ML Unige 2023: Advertisement Impact Prediction

Predicting customer subscriptions from an advertisement campaign

Liam Phan, Michael Bigler and Manuela Giansante

---

Goal

A company wants to assess the quality of their online advertisement campaign. Online users are the main interest in this campaign. The users see a web banner during their browsing activity. For each user, they want to be able to predict whether they subscribe to the advertised product through the advertisement banner, based on the information they have about them. To subscribe, the user has to click on the banner and then subscribe to the service.

The target variable name is subscription.

Packages

import pandas as pd
import numpy as np
from sklearn.preprocessing import StandardScaler
from sklearn.model_selection import train_test_split
from facets_overview.generic_feature_statistics_generator import GenericFeatureStatisticsGenerator
import matplotlib.pyplot as plt
import missingno as msno
import seaborn as sns
import matplotlib as mpl
import matplotlib.colors as mcolors

import pandas as pd
import matplotlib.pyplot as plt
import missingno as msno
import seaborn as sns
import numpy as np
import datetime
from sklearn.preprocessing import StandardScaler
from sklearn.preprocessing import OrdinalEncoder
from sklearn.experimental import enable_iterative_imputer
from sklearn.impute import KNNImputer, IterativeImputer, SimpleImputer
from sklearn.preprocessing import PolynomialFeatures
from sklearn.model_selection import train_test_split
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.metrics import RocCurveDisplay
from sklearn.metrics import confusion_matrix
from sklearn.neighbors import KNeighborsClassifier
from sklearn.model_selection import GridSearchCV, RandomizedSearchCV
from sklearn.model_selection import KFold
from sklearn.linear_model import RidgeClassifier, Lasso
from sklearn.linear_model import LogisticRegression
from sklearn.tree import DecisionTreeClassifier
import xgboost as xgb
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers

# Color palette
main_col = "#274156"
second_col = "#A2A77F"
target_col = "#499167"

Data

File descriptions

---

MLUnige2023_subscriptions_train.csv - The training set (features + response).

MLUnige2023_subscriptions_test.csv - The test set (features only).

MLUnige2023_sampleSubmission.csv - A sample submission file in the correct format (with random predictions).

Data fields

---

Unique identifier

1. Id: a unique identifier of the observations in each dataset. In the test set, it is used to match your predictions with the true values.

Target variable (only in the training data)

2. subscription: whether the user subscribed through the banner (1:yes, 0:no)

Demographic variables

3. age: (numeric)

4. job: type of job (categorical: teacher, industrial_worker, entrepreneur, housekeeper, manager, retired, freelance, salesman, student, technology, unemployed, na)

5. marital: marital status (categorical: married, divorced, single)

6. education: (categorical: high_school, university, grad_school, na)

Variables about the current campaign

7. device: from which device does the user see the banner? (categorical: smartphone, desktop, na)

8. day: last day of the month when the user saw the banner (numeric)

9. month: last month of the year when the user saw the banner (numeric)

10. time_spent: how long the user looked at the banner last time (in seconds) (numeric)

11. banner_views: number of times the user saw the banner (numeric)

Variables about an old campaign for the same product

12. banner_views_old: number of times the user saw the banner during an old (and related) online ads campaign (numeric)

13. days_elapsed_old: number of days since the user saw the banner of an old (and related) online ads campaign (numeric, -1 if the user never saw the banner)

14. outcome_old: outcome of the old (and related) online ads campaign (categorical: failure, other, success, na)

Variables with no name

15. X1: (categorical: 1, 0)

16. X2: (categorical: 1, 0)

17. X3: (categorical: 1, 0)

18. X4: (numeric)

---

We first start by loading both Datasets

# Train
train =  pd.read_csv('DATA/MLUnige2023_subscriptions_train.csv', index_col='Id', na_values='na', header=0)
train = train.astype({'job':'category', 'marital':'category', 'education':'category', 'device':'category', 'outcome_old':'category', 'X1':'category', 'X2':'category', 'X3':'category', 'X4':'float64', 'subscription':'category', 'day':'category', 'month':'category'})
train.day = pd.Categorical(train.day, ordered=True)
train.month = pd.Categorical(train.month, ordered=True)

train_categorical = list(train.select_dtypes(include=['category']).columns)
train_numeric = list(train.select_dtypes(include=['int64', 'float64']).columns)

train_numeric_target = train_numeric.copy()
train_numeric_target.append("subscription")

# Test
test =  pd.read_csv('DATA/MLUnige2023_subscriptions_test.csv', index_col='Id', na_values='na', header=0)
test = test.astype({'job':'category', 'marital':'category', 'education':'category', 'device':'category', 'outcome_old':'category', 'X1':'category', 'X2':'category', 'X3':'category', 'X4':'float64', 'day':'category', 'month':'category'})
test.day = pd.Categorical(test.day, ordered=True)
test.month = pd.Categorical(test.month, ordered=True)

test_categorical = list(test.select_dtypes(include=['category']).columns)
test_numeric = list(test.select_dtypes(include=['int64', 'float64']).columns)

Here is our Train Dataset:

train

	age	job	marital	education	device	day	month	time_spent	banner_views	banner_views_old	days_elapsed_old	outcome_old	X1	X2	X3	X4	subscription
Id
0	28	freelance	married	grad_school	smartphone	4	2	26.80	3	4	196	failure	0	0	1	0.072803	1
1	48	industrial_worker	married	university	smartphone	30	4	13.05	1	1	79	success	0	0	0	0.075454	1
2	27	teacher	married	university	smartphone	14	7	8.10	3	0	-1	NaN	0	1	1	0.068110	0
3	44	unemployed	divorced	university	smartphone	13	5	7.10	2	1	369	other	0	0	1	0.091942	0
4	29	manager	single	grad_school	smartphone	26	4	15.90	2	2	143	success	0	0	0	0.085922	1
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
8947	54	industrial_worker	married	university	smartphone	16	7	7.30	1	0	-1	NaN	1	0	0	0.072803	0
8948	43	industrial_worker	married	university	smartphone	4	2	37.75	2	0	-1	NaN	0	0	1	0.081456	1
8949	27	manager	single	grad_school	NaN	5	6	29.00	3	0	-1	NaN	0	0	1	0.079186	1
8950	77	retired	divorced	grad_school	smartphone	14	4	7.55	1	0	-1	NaN	0	0	0	0.115102	1
8951	33	technology	single	grad_school	smartphone	4	5	29.10	1	1	66	success	0	0	1	0.087656	1

8952 rows × 17 columns

print("We have",train.shape[0], "Rows and", train.shape[1],"Columns")

We have 8952 Rows and 17 Columns

Here is our Test Dataset:

test

	age	job	marital	education	device	day	month	time_spent	banner_views	banner_views_old	days_elapsed_old	outcome_old	X1	X2	X3	X4
Id
0	53	retired	married	university	NaN	17	6	5.25	1	0	-1	NaN	0	0	1	0.084570
1	61	manager	married	grad_school	smartphone	20	4	9.00	1	0	-1	NaN	0	0	0	0.075227
2	51	industrial_worker	married	university	NaN	4	7	9.65	1	0	-1	NaN	0	0	0	0.075781
3	34	manager	married	grad_school	NaN	28	5	13.45	2	0	-1	NaN	0	0	1	0.070043
4	30	manager	married	grad_school	smartphone	4	5	16.15	2	0	-1	NaN	0	0	0	0.171618
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
3832	47	industrial_worker	married	NaN	NaN	5	6	19.20	2	0	-1	NaN	0	0	1	0.104216
3833	30	teacher	single	university	smartphone	3	3	3.65	1	0	-1	NaN	1	0	0	0.077552
3834	55	teacher	married	university	NaN	26	6	5.15	5	0	-1	NaN	0	0	1	0.129156
3835	46	teacher	divorced	university	smartphone	13	2	5.55	1	0	-1	NaN	0	0	0	0.093067
3836	53	technology	married	university	desktop	17	4	15.50	3	1	180	success	0	0	1	0.120949

3837 rows × 16 columns

print("We have",test.shape[0], "Rows and", test.shape[1],"Columns (We don't know the subscription outcome of the test data)")

We have 3837 Rows and 16 Columns (We don't know the subscription outcome of the test data)

Exploratory Data Analysis

Duplicates

We check if there is any perfect duplicates in both our Datasets:

# Train
## check for duplicate rows
duplicates = train.duplicated()

## display the duplicate rows
print("\n",train[duplicates])

# Test
## check for duplicate rows
duplicates = test.duplicated()

## display the duplicate rows
print("\n",test[duplicates])

 Empty DataFrame
Columns: [age, job, marital, education, device, day, month, time_spent, banner_views, banner_views_old, days_elapsed_old, outcome_old, X1, X2, X3, X4, subscription]
Index: []

 Empty DataFrame
Columns: [age, job, marital, education, device, day, month, time_spent, banner_views, banner_views_old, days_elapsed_old, outcome_old, X1, X2, X3, X4]
Index: []

The dataframes Train and Test has no duplicate rows.

Missings

We then check how many and where are our missings values in both Train and Test.

Train

train.isna().sum()

age                    0
job                   61
marital                0
education            378
device              2050
day                    0
month                  0
time_spent             0
banner_views           0
banner_views_old       0
days_elapsed_old       0
outcome_old         6804
X1                     0
X2                     0
X3                     0
X4                     0
subscription           0
dtype: int64

Test

test.isna().sum()

age                    0
job                   21
marital                0
education            160
device               901
day                    0
month                  0
time_spent             0
banner_views           0
banner_views_old       0
days_elapsed_old       0
outcome_old         2937
X1                     0
X2                     0
X3                     0
X4                     0
dtype: int64

In both Datasets, we have a lot of missings in outcome_old, device, education and job

We also know that day_elapsed_old is having the same missings as outcome_old, just encoded as -1 instead. We can also visualize it with a plot:

Train

mpl.rcParams['figure.dpi']= 300

msno.bar(train,color=main_col, sort="ascending")
plt.title("Missing Values on Train", fontweight='bold', fontsize=24)

Text(0.5, 1.0, 'Missing Values on Train')

Test

mpl.rcParams['figure.dpi']= 300

msno.bar(test,color=second_col, sort="ascending")
plt.title("Missing Values on Test", fontweight='bold', fontsize=24)

Text(0.5, 1.0, 'Missing Values on Test')

We better see that outcome_old and device are our main missings with education (and day_elapsed_old is -1 but still would count as a theorical missing)

Visualization

Pairplot

mpl.rcParams['figure.dpi']= 300

colors_subscription = [main_col, second_col]
sns.pairplot(train, hue = "subscription", palette=colors_subscription)

<seaborn.axisgrid.PairGrid object at 0x2f148dc90>

Barplots

mpl.rcParams['figure.dpi']= 300

sns.countplot(x="subscription", data=train,palette= colors_subscription)

plt.title("Subscription", fontsize=18)
plt.yticks(fontsize=6)

(array([   0., 1000., 2000., 3000., 4000., 5000., 6000.]), [Text(0, 0.0, '0'), Text(0, 1000.0, '1000'), Text(0, 2000.0, '2000'), Text(0, 3000.0, '3000'), Text(0, 4000.0, '4000'), Text(0, 5000.0, '5000'), Text(0, 6000.0, '6000')])

sns.countplot(x="job", hue="subscription", data=train, palette=colors_subscription)

plt.xticks(rotation=45)
plt.title("Type of Jobs VS Subscription", fontsize=18)

plt.xticks(fontsize=6)
plt.yticks(fontsize=6)

(array([   0.,  200.,  400.,  600.,  800., 1000., 1200., 1400.]), [Text(0, 0.0, '0'), Text(0, 200.0, '200'), Text(0, 400.0, '400'), Text(0, 600.0, '600'), Text(0, 800.0, '800'), Text(0, 1000.0, '1000'), Text(0, 1200.0, '1200'), Text(0, 1400.0, '1400')])

sns.countplot(x="marital", hue="subscription", data=train, palette=colors_subscription)

plt.xticks(rotation=45)
plt.title("Marital Status VS Subscription", fontsize=18)

plt.xticks(fontsize=8)
plt.yticks(fontsize=6)

(array([   0.,  500., 1000., 1500., 2000., 2500., 3000., 3500.]), [Text(0, 0.0, '0'), Text(0, 500.0, '500'), Text(0, 1000.0, '1000'), Text(0, 1500.0, '1500'), Text(0, 2000.0, '2000'), Text(0, 2500.0, '2500'), Text(0, 3000.0, '3000'), Text(0, 3500.0, '3500')])

sns.countplot(x="education", hue="subscription", data=train, palette=colors_subscription)

plt.xticks(rotation=45)
plt.title("Education VS Subscription", fontsize=18)

plt.xticks(fontsize=8)
plt.yticks(fontsize=6)

(array([   0.,  500., 1000., 1500., 2000., 2500., 3000.]), [Text(0, 0.0, '0'), Text(0, 500.0, '500'), Text(0, 1000.0, '1000'), Text(0, 1500.0, '1500'), Text(0, 2000.0, '2000'), Text(0, 2500.0, '2500'), Text(0, 3000.0, '3000')])

sns.countplot(x="day", hue="subscription", data=train, palette=colors_subscription)

plt.title("Day VS Subscription", fontsize=18)

plt.xticks(fontsize=6)
plt.yticks(fontsize=6)

(array([  0.,  50., 100., 150., 200., 250., 300., 350., 400.]), [Text(0, 0.0, '0'), Text(0, 50.0, '50'), Text(0, 100.0, '100'), Text(0, 150.0, '150'), Text(0, 200.0, '200'), Text(0, 250.0, '250'), Text(0, 300.0, '300'), Text(0, 350.0, '350'), Text(0, 400.0, '400')])

sns.countplot(x="month", hue="subscription", data=train, palette=colors_subscription)

plt.title("Month VS Subscription", fontsize=16)

plt.xticks(fontsize=6)
plt.yticks(fontsize=6)

(array([   0.,  250.,  500.,  750., 1000., 1250., 1500., 1750., 2000.]), [Text(0, 0.0, '0'), Text(0, 250.0, '250'), Text(0, 500.0, '500'), Text(0, 750.0, '750'), Text(0, 1000.0, '1000'), Text(0, 1250.0, '1250'), Text(0, 1500.0, '1500'), Text(0, 1750.0, '1750'), Text(0, 2000.0, '2000')])

f, axes = plt.subplots(1, 3)

sns.countplot(x="X1", hue="subscription", data=train, palette=colors_subscription, ax=axes[0])
sns.countplot(x="X2", hue="subscription", data=train, palette=colors_subscription, ax=axes[1])
sns.countplot(x="X3", hue="subscription", data=train, palette=colors_subscription, ax=axes[2])

# Set the title for the figure
plt.suptitle("X1,X2 and X3 VS Subscription ", fontsize=20)

# Set the figure size
fig = plt.gcf()
fig.set_size_inches(12, 8)

# Adjust the spacing between subplots
plt.subplots_adjust(wspace=0.3)

# Show the combined bar plots
plt.show()

Correlation Map

mpl.rcParams['figure.dpi']= 200

custom_cmap = mcolors.LinearSegmentedColormap.from_list("", [second_col, "white", main_col])

# Calculate the correlation matrix using pandas
corr_matrix = train.corr()

# Create a figure with a specific size using plt.subplots()
fig, ax = plt.subplots(figsize=(10, 8))

# Create a heatmap of the correlation matrix using sns.heatmap()
sns.heatmap(corr_matrix, annot=True, cmap=custom_cmap, fmt='.2f', center=0)

# Add a title to the plot
ax.set_title("Correlation Map", fontsize=20)

# Show the plot
plt.show()

/var/folders/j0/m71dtf9j0rvddgqlps9swgxc0000gn/T/ipykernel_41286/1257374829.py:6: FutureWarning: The default value of numeric_only in DataFrame.corr is deprecated. In a future version, it will default to False. Select only valid columns or specify the value of numeric_only to silence this warning.
  corr_matrix = train.corr()

We don't have strong correlations among our set of numeric variables, mostly under 0.7 or above -0.7. Therefore there is no risk of multicolinearity.

Data Featuring and Engineering

Scaling

train_num = train[train_numeric]
train_cat = train[train_categorical]

test_num = test[test_numeric]
test_cat = test[test_categorical]

# create a StandardScaler object
scaler = StandardScaler()

# fit and transform the scaler on the numerical variables
scaler.fit(train_num)
train_num = pd.DataFrame(scaler.transform(train_num), columns=train_num.columns, index=train_num.index)
test_num = pd.DataFrame(scaler.transform(test_num),columns=test_num.columns, index=test_num.index)

# merge again 
train_scale = pd.merge(train_num, train_cat, how="left", on=["Id"])
test_scale = pd.merge(test_num, test_cat, how="left", on=["Id"])

Train

train_scale.head()

	age	time_spent	banner_views	banner_views_old	days_elapsed_old	X4	job	marital	education	device	day	month	outcome_old	X1	X2	X3	subscription
Id
0	-1.134397	0.572110	0.162700	1.543875	1.365471	-0.456189	freelance	married	grad_school	smartphone	4	2	failure	0	0	1	1
1	0.584160	-0.241413	-0.545538	0.114732	0.275678	-0.368554	industrial_worker	married	university	smartphone	30	4	success	0	0	0	1
2	-1.220325	-0.534281	0.162700	-0.361649	-0.469480	-0.611350	teacher	married	university	smartphone	14	7	NaN	0	1	1	0
3	0.240448	-0.593446	-0.191419	0.114732	2.976874	0.176463	unemployed	divorced	university	smartphone	13	5	other	0	0	1	0
4	-1.048469	-0.072791	-0.191419	0.591113	0.871804	-0.022516	manager	single	grad_school	smartphone	26	4	success	0	0	0	1

Test

test_scale.head()

	age	time_spent	banner_views	banner_views_old	days_elapsed_old	X4	job	marital	education	device	day	month	outcome_old	X1	X2	X3
Id
0	1.013799	-0.702902	-0.545538	-0.361649	-0.46948	-0.067234	retired	married	university	NaN	17	6	NaN	0	0	1
1	1.701222	-0.481032	-0.545538	-0.361649	-0.46948	-0.376057	manager	married	grad_school	smartphone	20	4	NaN	0	0	0
2	0.841943	-0.442575	-0.545538	-0.361649	-0.46948	-0.357749	industrial_worker	married	university	NaN	4	7	NaN	0	0	0
3	-0.618830	-0.217746	-0.191419	-0.361649	-0.46948	-0.547425	manager	married	grad_school	NaN	28	5	NaN	0	0	1
4	-0.962542	-0.058000	-0.191419	-0.361649	-0.46948	2.810310	manager	married	grad_school	smartphone	4	5	NaN	0	0	0

Hot-Encoding

train_hot = pd.get_dummies(train_scale, columns=['job','marital','education','device','day','month','outcome_old'])
test_hot = pd.get_dummies(test_scale, columns=['job','marital','education','device','day','month','outcome_old'])

Train

train_hot = train_hot.astype({'X1':'uint8', 'X2':'uint8', 'X3':'uint8'})
train_hot.head()

	age	time_spent	banner_views	banner_views_old	days_elapsed_old	X4	X1	X2	X3	subscription	...	month_6	month_7	month_8	month_9	month_10	month_11	month_12	outcome_old_failure	outcome_old_other	outcome_old_success
Id
0	-1.134397	0.572110	0.162700	1.543875	1.365471	-0.456189	0	0	1	1	...	0	0	0	0	0	0	0	1	0	0
1	0.584160	-0.241413	-0.545538	0.114732	0.275678	-0.368554	0	0	0	1	...	0	0	0	0	0	0	0	0	0	1
2	-1.220325	-0.534281	0.162700	-0.361649	-0.469480	-0.611350	0	1	1	0	...	0	1	0	0	0	0	0	0	0	0
3	0.240448	-0.593446	-0.191419	0.114732	2.976874	0.176463	0	0	1	0	...	0	0	0	0	0	0	0	0	1	0
4	-1.048469	-0.072791	-0.191419	0.591113	0.871804	-0.022516	0	0	0	1	...	0	0	0	0	0	0	0	0	0	1

5 rows × 75 columns

Test

test_hot = test_hot.astype({'X1':'uint8', 'X2':'uint8', 'X3':'uint8'})
test_hot.head()

	age	time_spent	banner_views	banner_views_old	days_elapsed_old	X4	X1	X2	X3	job_entrepreneur	...	month_6	month_7	month_8	month_9	month_10	month_11	month_12	outcome_old_failure	outcome_old_other	outcome_old_success
Id
0	1.013799	-0.702902	-0.545538	-0.361649	-0.46948	-0.067234	0	0	1	0	...	1	0	0	0	0	0	0	0	0	0
1	1.701222	-0.481032	-0.545538	-0.361649	-0.46948	-0.376057	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0
2	0.841943	-0.442575	-0.545538	-0.361649	-0.46948	-0.357749	0	0	0	0	...	0	1	0	0	0	0	0	0	0	0
3	-0.618830	-0.217746	-0.191419	-0.361649	-0.46948	-0.547425	0	0	1	0	...	0	0	0	0	0	0	0	0	0	0
4	-0.962542	-0.058000	-0.191419	-0.361649	-0.46948	2.810310	0	0	0	0	...	0	0	0	0	0	0	0	0	0	0

5 rows × 74 columns

Train and Validation

Seperating Features and Target Variable

train_no_subscription = train_hot.drop('subscription', axis=1)
train_only_subscription = train_hot['subscription']

Splitting Training and Validation

valid_size = 0.23

# Split the data into training and test sets
X_train, X_valid, Y_train, Y_valid = train_test_split(train_no_subscription, train_only_subscription, test_size=valid_size,  stratify=train_only_subscription)

# Checking Correct Split
Cond1 = X_train.shape[0] == Y_train.shape[0]
Cond2 = X_valid.shape[0] == Y_valid.shape[0]

#print("\nSame Size ?","\n\nFor Training:",Cond1,", For Validation:", Cond2)

PropTV = (X_valid.shape[0]/X_train.shape[0])*100
PropTR = (1-(X_valid.shape[0]/X_train.shape[0]))*100

#print("\nProportions between Training and Validation:","\n Training: ", round(PropTR,2),"%", "\n Validation: ",round(PropTV,2),"%")

mpl.rcParams['figure.dpi']= 200

mylabels = ["Training","Validation"]

y = np.array([PropTR,PropTV])

piecolors = [second_col,main_col]

plt.pie(y, labels = mylabels, colors = piecolors, autopct='%1.1f%%')
plt.show() 

Predictive Models

mpl.rcParams['figure.dpi']= 150

custom_cmap2 = mcolors.LinearSegmentedColormap.from_list("", [main_col, "white", second_col])

def plot_ROC(true, predicted):
    RocCurveDisplay.from_predictions(
        true,
        predicted,
        color=second_col,
    )
    plt.plot([0, 1], [0, 1], "k--", label="chance level (AUC = 0.5)")
    plt.axis("square")
    plt.xlabel("False Positive Rate")
    plt.ylabel("True Positive Rate")
    plt.legend()
    plt.show()


def make_confusion_matrix(cf,
                          group_names= None,
                          categories= 'auto',
                          count=True,
                          percent=True,
                          cbar=True,
                          xyticks=True,
                          xyplotlabels=True,
                          sum_stats=True,
                          figsize=None,
                          cmap='Blues',
                          title=None):
    '''
    This function will make a pretty plot of an sklearn Confusion Matrix cm using a Seaborn heatmap visualization.

    Arguments
    ---------
    cf:            confusion matrix to be passed in

    group_names:   List of strings that represent the labels row by row to be shown in each square.

    categories:    List of strings containing the categories to be displayed on the x,y axis. Default is 'auto'

    count:         If True, show the raw number in the confusion matrix. Default is True.

    normalize:     If True, show the proportions for each category. Default is True.

    cbar:          If True, show the color bar. The cbar values are based off the values in the confusion matrix.
                   Default is True.

    xyticks:       If True, show x and y ticks. Default is True.

    xyplotlabels:  If True, show 'True Label' and 'Predicted Label' on the figure. Default is True.

    sum_stats:     If True, display summary statistics below the figure. Default is True.

    figsize:       Tuple representing the figure size. Default will be the matplotlib rcParams value.

    cmap:          Colormap of the values displayed from matplotlib.pyplot.cm. Default is 'Blues'
                   See http://matplotlib.org/examples/color/colormaps_reference.html
                   
    title:         Title for the heatmap. Default is None.

    '''


    # CODE TO GENERATE TEXT INSIDE EACH SQUARE
    blanks = ['' for i in range(cf.size)]

    if group_names and len(group_names)==cf.size:
        group_labels = ["{}\n".format(value) for value in group_names]
    else:
        group_labels = blanks

    if count:
        group_counts = ["{0:0.0f}\n".format(value) for value in cf.flatten()]
    else:
        group_counts = blanks

    if percent:
        group_percentages = ["{0:.2%}".format(value) for value in cf.flatten()/np.sum(cf)]
    else:
        group_percentages = blanks

    box_labels = [f"{v1}{v2}{v3}".strip() for v1, v2, v3 in zip(group_labels,group_counts,group_percentages)]
    box_labels = np.asarray(box_labels).reshape(cf.shape[0],cf.shape[1])


    # CODE TO GENERATE SUMMARY STATISTICS & TEXT FOR SUMMARY STATS
    if sum_stats:
        #Accuracy is sum of diagonal divided by total observations
        accuracy  = np.trace(cf) / float(np.sum(cf))

        #if it is a binary confusion matrix, show some more stats
        if len(cf)==2:
            #Metrics for Binary Confusion Matrices
            precision = cf[1,1] / sum(cf[:,1])
            recall    = cf[1,1] / sum(cf[1,:])
            f1_score  = 2*precision*recall / (precision + recall)
            specificity = cf[0,0] / (cf[0,0] + cf[0,1])
            sensitivity = cf[1,1] / (cf[1,0] + cf[1,1])
            stats_text = "\n\nAccuracy={:0.3f}   Precision={:0.3f}   Recall={:0.3f}\n      F1 Score={:0.3f}   Sensitivity{:0.3f}   Specificity={:0.3f}".format(
                accuracy,precision,recall,f1_score, sensitivity,specificity)
        else:
            stats_text = "\n\nAccuracy={:0.3f}".format(accuracy)
    else:
        stats_text = ""


    # SET FIGURE PARAMETERS ACCORDING TO OTHER ARGUMENTS
    if figsize==None:
        #Get default figure size if not set
        figsize = plt.rcParams.get('figure.figsize')

    if xyticks==False:
        #Do not show categories if xyticks is False
        categories=False


    # MAKE THE HEATMAP VISUALIZATION
    plt.figure(figsize=figsize)
    sns.heatmap(cf,annot=box_labels,fmt="",cmap=cmap,cbar=cbar,xticklabels=categories,yticklabels=categories)

    if xyplotlabels:
        plt.ylabel('True label')
        plt.xlabel('Predicted label' + stats_text)
    else:
        plt.xlabel(stats_text)
    
    if title:
        plt.title(title)
    
    plt.show()


def plot_training(history_dict):
    acc = history_dict['accuracy']
    val_acc = history_dict['val_accuracy']
    loss = history_dict['loss']
    val_loss = history_dict['val_loss']
    
    epochs = range(1, len(loss) + 1)
    
    plt.figure(figsize=(10, 4))
    plt.subplot(1, 2, 1)
    plt.plot(epochs, loss, 'b--', label='Training')
    plt.plot(epochs, val_loss, 'r-', label='Validation')
    plt.xlabel('Epochs')
    plt.ylabel('Cross-Entropy Loss')
    plt.legend()
    
    plt.subplot(1, 2, 2)
    plt.plot(epochs, acc, 'b--', label='Training')
    plt.plot(epochs, val_acc, 'r-', label='Validation')
    plt.xlabel('Epochs')
    plt.ylabel('Accuracy')
    plt.legend(loc='lower right')
    
    plt.show()

Random Forests

from sklearn.ensemble import RandomForestClassifier

No Tuning

# create the Random Forest model
rf = RandomForestClassifier(n_estimators=100, random_state=42)

# fit the model to the training data
rf.fit(X_train, Y_train)

# make predictions on the testing data
y_pred = rf.predict(X_valid)

make_confusion_matrix(confusion_matrix(Y_valid, y_pred), group_names=['True Neg','False Pos','False Neg','True Pos'],  categories=['Zero', 'One'], cmap=custom_cmap2)
plot_ROC(Y_valid, y_pred)

Tuning

rf = RandomForestClassifier(random_state=42)

# set up the hyperparameter grid to search over
param_grid = {'n_estimators': [50, 100, 200],
              'max_depth': [None, 5, 10],
              'min_samples_split': [2, 5, 10],
              'min_samples_leaf': [1, 2, 4]}

# set up the GridSearchCV object
grid_search = GridSearchCV(rf, param_grid, cv=5)

# fit the GridSearchCV object to the training data
grid_search.fit(X_train, Y_train)


rf = RandomForestClassifier(n_estimators=grid_search.best_params_['n_estimators'],
                            max_depth=grid_search.best_params_['max_depth'],
                            min_samples_split=grid_search.best_params_['min_samples_split'],
                            min_samples_leaf=grid_search.best_params_['min_samples_leaf'],
                            random_state=42)

# fit the model to the entire training data
rf.fit(X_train, Y_train)

# make predictions on the testing data
y_pred = rf.predict(X_valid)

make_confusion_matrix(confusion_matrix(Y_valid, y_pred), group_names=['True Neg','False Pos','False Neg','True Pos'],  categories=['Zero', 'One'], cmap=custom_cmap2)
plot_ROC(Y_valid, y_pred)

Bagging Trees

from sklearn.ensemble import BaggingClassifier
from sklearn.tree import DecisionTreeClassifier

No Tuning

# create the Decision Tree model
dt = DecisionTreeClassifier(random_state=42)

# create the Bagging model with 100 base estimators
bagging = BaggingClassifier(dt, n_estimators=100, random_state=42)

# fit the model to the training data
bagging.fit(X_train, Y_train)

# make predictions on the testing data
y_pred = bagging.predict(X_valid)

make_confusion_matrix(confusion_matrix(Y_valid, y_pred), group_names=['True Neg','False Pos','False Neg','True Pos'],  categories=['Zero', 'One'], cmap=custom_cmap2)
plot_ROC(Y_valid, y_pred)

Tuning

param_grid = {'n_estimators': [50, 100, 200],
              'max_samples': [0.5, 1.0],
              'bootstrap': [True, False],
              'bootstrap_features': [True, False]}

# create the GridSearchCV object
grid_search = GridSearchCV(BaggingClassifier(DecisionTreeClassifier(random_state=42), random_state=42),
                           param_grid, cv=5)

# fit the GridSearchCV object to the training data
grid_search.fit(X_train, Y_train)

# create a new Bagging model with the best hyperparameters found during the hyperparameter tuning process
bagging = BaggingClassifier(DecisionTreeClassifier(random_state=42),
                            n_estimators=grid_search.best_params_['n_estimators'],
                            max_samples=grid_search.best_params_['max_samples'],
                            bootstrap=grid_search.best_params_['bootstrap'],
                            bootstrap_features=grid_search.best_params_['bootstrap_features'],
                            random_state=42)

# fit the model to the entire training data
bagging.fit(X_train, Y_train)

y_pred = bagging.predict(X_valid)

make_confusion_matrix(confusion_matrix(Y_valid, y_pred), group_names=['True Neg','False Pos','False Neg','True Pos'],  categories=['Zero', 'One'], cmap=custom_cmap2)
plot_ROC(Y_valid, y_pred)

Boosting Trees

from sklearn.ensemble import GradientBoostingClassifier

No Tuning

gb = GradientBoostingClassifier(random_state=42)

# fit the model to the training data
gb.fit(X_train, Y_train)

# make predictions on the testing data
y_pred = gb.predict(X_valid)

make_confusion_matrix(confusion_matrix(Y_valid, y_pred), group_names=['True Neg','False Pos','False Neg','True Pos'],  categories=['Zero', 'One'], cmap=custom_cmap2)
plot_ROC(Y_valid, y_pred)

Tuning

gb = GradientBoostingClassifier(random_state=42)

# set up the hyperparameter grid to search over
param_grid = {'n_estimators': [50, 100, 200],
              'learning_rate': [0.01, 0.1, 1.0],
              'max_depth': [2, 3, 4],
              'subsample': [0.5, 0.75, 1.0]}

# create the GridSearchCV object
grid_search = GridSearchCV(gb, param_grid, cv=5)

# fit the GridSearchCV object to the training data
grid_search.fit(X_train, Y_train)

# print the best hyperparameters found during the hyperparameter tuning process
print("Best hyperparameters: ", grid_search.best_params_)

# create a new Gradient Boosting model with the best hyperparameters found during the hyperparameter tuning process
gb = GradientBoostingClassifier(n_estimators=grid_search.best_params_['n_estimators'],
                                learning_rate=grid_search.best_params_['learning_rate'],
                                max_depth=grid_search.best_params_['max_depth'],
                                subsample=grid_search.best_params_['subsample'],
                                random_state=42)

# fit the model to the entire training data
gb.fit(X_train, Y_train)

# make predictions on the testing data
y_pred = gb.predict(X_valid)

make_confusion_matrix(confusion_matrix(Y_valid, y_pred), group_names=['True Neg','False Pos','False Neg','True Pos'],  categories=['Zero', 'One'], cmap=custom_cmap2)
plot_ROC(Y_valid, y_pred)

Best hyperparameters:  {'learning_rate': 0.1, 'max_depth': 4, 'n_estimators': 200, 'subsample': 1.0}

Linear Discriminant Analysis (LDA)

from sklearn.discriminant_analysis import LinearDiscriminantAnalysis

lda = LinearDiscriminantAnalysis()

# fit the LDA model to the training data
lda.fit(X_train, Y_train)

# make predictions on the testing data using the trained LDA model
y_pred = lda.predict(X_valid)

make_confusion_matrix(confusion_matrix(Y_valid, y_pred), group_names=['True Neg','False Pos','False Neg','True Pos'],  categories=['Zero', 'One'],cmap=custom_cmap2)
plot_ROC(Y_valid, y_pred)

Support Vector Machines (SVM)

from sklearn import svm

No Tuning

svm_model = svm.SVC()

# fit the SVM model to the training data
svm_model.fit(X_train, Y_train)

# make predictions on the testing data using the trained SVM model
y_pred = svm_model.predict(X_valid)

make_confusion_matrix(confusion_matrix(Y_valid, y_pred), group_names=['True Neg','False Pos','False Neg','True Pos'],  categories=['Zero', 'One'], cmap=custom_cmap2)
plot_ROC(Y_valid, y_pred)

Tuning

svm_model = svm.SVC()

# set up the hyperparameter grid to search over
param_grid = {'C': [0.1, 1, 10, 100],
              'kernel': ['linear', 'poly', 'rbf', 'sigmoid'],
              'gamma': ['scale', 'auto']}

# create the GridSearchCV object
grid_search = GridSearchCV(svm_model, param_grid, cv=5)

# fit the GridSearchCV object to the training data
grid_search.fit(X_train, Y_train)

# print the best hyperparameters found during the hyperparameter tuning process
print("Best hyperparameters: ", grid_search.best_params_)

# create a new SVM model with the best hyperparameters found during thehyperparameter tuning process
svm_model = svm.SVC(C=grid_search.best_params_['C'],
                    kernel=grid_search.best_params_['kernel'],
                    gamma=grid_search.best_params_['gamma'])

# fit the SVM model to the training data
svm_model.fit(X_train, Y_train)

# make predictions on the testing data using the trained SVM model
y_pred = svm_model.predict(X_valid)

make_confusion_matrix(confusion_matrix(Y_valid, y_pred), group_names=['True Neg','False Pos','False Neg','True Pos'],  categories=['Zero', 'One'], cmap=custom_cmap2)
plot_ROC(Y_valid, y_pred)

Best hyperparameters:  {'C': 1, 'gamma': 'scale', 'kernel': 'rbf'}

XGBoost

import xgboost as xgb
from xgboost import XGBClassifier
from sklearn.metrics import accuracy_score
from matplotlib import pyplot

No Tuning

# Instantiate the XGBClassifier class
xg_clf = xgb.XGBClassifier(objective='binary:logistic', n_estimators=10, seed=123)

# Fit the model on the training data
xg_clf.fit(X_train, Y_train)

# Make predictions on the test data
y_pred = xg_clf.predict(X_valid)

make_confusion_matrix(confusion_matrix(Y_valid, y_pred), group_names=['True Neg','False Pos','False Neg','True Pos'],  categories=['Zero', 'One'], cmap=custom_cmap2)
plot_ROC(Y_valid, y_pred)

Tuning V1

# fit model no training data
model = XGBClassifier()
eval_set = [(X_train, Y_train), (X_valid, Y_valid)]
model.fit(X_train, Y_train, eval_metric=["error", "logloss"], eval_set=eval_set, verbose=True)
# make predictions for test data
y_pred = model.predict(X_valid)
predictions = [round(value) for value in y_pred]
# evaluate predictions
accuracy = accuracy_score(Y_valid, predictions)
# retrieve performance metrics
results = model.evals_result()
epochs = len(results['validation_0']['error'])
x_axis = range(0, epochs)
# plot log loss
fig, ax = pyplot.subplots()
ax.plot(x_axis, results['validation_0']['logloss'], color=second_col,label='Train')
ax.plot(x_axis, results['validation_1']['logloss'], color=main_col,label='Validation')
ax.legend()
pyplot.ylabel('Log Loss')
pyplot.title('XGBoost Log Loss')
pyplot.show()
# plot classification error
fig, ax = pyplot.subplots()
ax.plot(x_axis, results['validation_0']['error'],color=second_col, label='Train')
ax.plot(x_axis, results['validation_1']['error'], color=main_col,label='Validation')
ax.legend()
pyplot.ylabel('Classification Error')
pyplot.title('XGBoost Classification Error')
pyplot.show()

make_confusion_matrix(confusion_matrix(Y_valid, y_pred), group_names=['True Neg','False Pos','False Neg','True Pos'],  categories=['Zero', 'One'], cmap=custom_cmap2)
plot_ROC(Y_valid, y_pred)

[0]	validation_0-error:0.17656	validation_0-logloss:0.56468	validation_1-error:0.20204	validation_1-logloss:0.57498
[1]	validation_0-error:0.15233	validation_0-logloss:0.48906	validation_1-error:0.16416	validation_1-logloss:0.50466
[2]	validation_0-error:0.14812	validation_0-logloss:0.43840	validation_1-error:0.16367	validation_1-logloss:0.46187
[3]	validation_0-error:0.14711	validation_0-logloss:0.40432	validation_1-error:0.16513	validation_1-logloss:0.43517
[4]	validation_0-error:0.14043	validation_0-logloss:0.37727	validation_1-error:0.16319	validation_1-logloss:0.41418
[5]	validation_0-error:0.13724	validation_0-logloss:0.35845	validation_1-error:0.15979	validation_1-logloss:0.39998
[6]	validation_0-error:0.13434	validation_0-logloss:0.34262	validation_1-error:0.15930	validation_1-logloss:0.38656
[7]	validation_0-error:0.13100	validation_0-logloss:0.32903	validation_1-error:0.15833	validation_1-logloss:0.37659
[8]	validation_0-error:0.12999	validation_0-logloss:0.31925	validation_1-error:0.15639	validation_1-logloss:0.37091
[9]	validation_0-error:0.12912	validation_0-logloss:0.31133	validation_1-error:0.15639	validation_1-logloss:0.36666
[10]	validation_0-error:0.12462	validation_0-logloss:0.30229	validation_1-error:0.15299	validation_1-logloss:0.36076
[11]	validation_0-error:0.12230	validation_0-logloss:0.29495	validation_1-error:0.14959	validation_1-logloss:0.35852
[12]	validation_0-error:0.11925	validation_0-logloss:0.29056	validation_1-error:0.15153	validation_1-logloss:0.35532
[13]	validation_0-error:0.11940	validation_0-logloss:0.28444	validation_1-error:0.14862	validation_1-logloss:0.35271
[14]	validation_0-error:0.11504	validation_0-logloss:0.27814	validation_1-error:0.14424	validation_1-logloss:0.34980
[15]	validation_0-error:0.11156	validation_0-logloss:0.27016	validation_1-error:0.14182	validation_1-logloss:0.34696
[16]	validation_0-error:0.11026	validation_0-logloss:0.26627	validation_1-error:0.14376	validation_1-logloss:0.34589
[17]	validation_0-error:0.10895	validation_0-logloss:0.26276	validation_1-error:0.14279	validation_1-logloss:0.34448
[18]	validation_0-error:0.10619	validation_0-logloss:0.25811	validation_1-error:0.14279	validation_1-logloss:0.34284
[19]	validation_0-error:0.10532	validation_0-logloss:0.25503	validation_1-error:0.14376	validation_1-logloss:0.34335
[20]	validation_0-error:0.10460	validation_0-logloss:0.25200	validation_1-error:0.14376	validation_1-logloss:0.34293
[21]	validation_0-error:0.10271	validation_0-logloss:0.25013	validation_1-error:0.14279	validation_1-logloss:0.34140
[22]	validation_0-error:0.10271	validation_0-logloss:0.24736	validation_1-error:0.14279	validation_1-logloss:0.34181
[23]	validation_0-error:0.10126	validation_0-logloss:0.24501	validation_1-error:0.14230	validation_1-logloss:0.34141
[24]	validation_0-error:0.09909	validation_0-logloss:0.24154	validation_1-error:0.14133	validation_1-logloss:0.33960
[25]	validation_0-error:0.09938	validation_0-logloss:0.23941	validation_1-error:0.14085	validation_1-logloss:0.34057
[26]	validation_0-error:0.09880	validation_0-logloss:0.23816	validation_1-error:0.13793	validation_1-logloss:0.34033
[27]	validation_0-error:0.09880	validation_0-logloss:0.23627	validation_1-error:0.13599	validation_1-logloss:0.34168
[28]	validation_0-error:0.09735	validation_0-logloss:0.23385	validation_1-error:0.13890	validation_1-logloss:0.33963

/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `eval_metric` in `fit` method is deprecated for better compatibility with scikit-learn, use `eval_metric` in constructor or`set_params` instead.
  warnings.warn(

[29]	validation_0-error:0.09560	validation_0-logloss:0.23059	validation_1-error:0.13890	validation_1-logloss:0.33934
[30]	validation_0-error:0.09517	validation_0-logloss:0.22902	validation_1-error:0.13939	validation_1-logloss:0.33889
[31]	validation_0-error:0.09415	validation_0-logloss:0.22666	validation_1-error:0.13793	validation_1-logloss:0.33961
[32]	validation_0-error:0.09154	validation_0-logloss:0.22524	validation_1-error:0.13745	validation_1-logloss:0.33923
[33]	validation_0-error:0.09140	validation_0-logloss:0.22246	validation_1-error:0.13890	validation_1-logloss:0.33906
[34]	validation_0-error:0.08922	validation_0-logloss:0.21907	validation_1-error:0.14036	validation_1-logloss:0.33843
[35]	validation_0-error:0.08748	validation_0-logloss:0.21605	validation_1-error:0.14182	validation_1-logloss:0.33698
[36]	validation_0-error:0.08646	validation_0-logloss:0.21439	validation_1-error:0.14230	validation_1-logloss:0.33693
[37]	validation_0-error:0.08603	validation_0-logloss:0.21361	validation_1-error:0.14133	validation_1-logloss:0.33693
[38]	validation_0-error:0.08371	validation_0-logloss:0.21055	validation_1-error:0.13987	validation_1-logloss:0.33607
[39]	validation_0-error:0.08298	validation_0-logloss:0.20940	validation_1-error:0.14085	validation_1-logloss:0.33644
[40]	validation_0-error:0.07892	validation_0-logloss:0.20622	validation_1-error:0.14230	validation_1-logloss:0.33585
[41]	validation_0-error:0.07849	validation_0-logloss:0.20496	validation_1-error:0.14279	validation_1-logloss:0.33624
[42]	validation_0-error:0.07689	validation_0-logloss:0.20378	validation_1-error:0.14230	validation_1-logloss:0.33692
[43]	validation_0-error:0.07558	validation_0-logloss:0.20194	validation_1-error:0.14230	validation_1-logloss:0.33794
[44]	validation_0-error:0.07486	validation_0-logloss:0.20106	validation_1-error:0.14279	validation_1-logloss:0.33885
[45]	validation_0-error:0.07239	validation_0-logloss:0.19895	validation_1-error:0.14230	validation_1-logloss:0.33834
[46]	validation_0-error:0.07138	validation_0-logloss:0.19694	validation_1-error:0.14230	validation_1-logloss:0.33867
[47]	validation_0-error:0.06978	validation_0-logloss:0.19551	validation_1-error:0.14279	validation_1-logloss:0.33906
[48]	validation_0-error:0.06819	validation_0-logloss:0.19214	validation_1-error:0.14182	validation_1-logloss:0.33960
[49]	validation_0-error:0.06775	validation_0-logloss:0.18980	validation_1-error:0.14279	validation_1-logloss:0.34054
[50]	validation_0-error:0.06630	validation_0-logloss:0.18804	validation_1-error:0.14327	validation_1-logloss:0.34004
[51]	validation_0-error:0.06470	validation_0-logloss:0.18414	validation_1-error:0.14327	validation_1-logloss:0.34087
[52]	validation_0-error:0.06441	validation_0-logloss:0.18264	validation_1-error:0.14230	validation_1-logloss:0.34116
[53]	validation_0-error:0.06383	validation_0-logloss:0.18131	validation_1-error:0.14182	validation_1-logloss:0.34262
[54]	validation_0-error:0.06195	validation_0-logloss:0.17902	validation_1-error:0.14327	validation_1-logloss:0.34275
[55]	validation_0-error:0.06166	validation_0-logloss:0.17854	validation_1-error:0.14327	validation_1-logloss:0.34275
[56]	validation_0-error:0.06195	validation_0-logloss:0.17776	validation_1-error:0.14327	validation_1-logloss:0.34234
[57]	validation_0-error:0.06122	validation_0-logloss:0.17668	validation_1-error:0.14279	validation_1-logloss:0.34291
[58]	validation_0-error:0.06021	validation_0-logloss:0.17552	validation_1-error:0.14182	validation_1-logloss:0.34289
[59]	validation_0-error:0.05890	validation_0-logloss:0.17363	validation_1-error:0.14570	validation_1-logloss:0.34307
[60]	validation_0-error:0.05716	validation_0-logloss:0.17155	validation_1-error:0.14473	validation_1-logloss:0.34394
[61]	validation_0-error:0.05600	validation_0-logloss:0.16969	validation_1-error:0.14376	validation_1-logloss:0.34417
[62]	validation_0-error:0.05440	validation_0-logloss:0.16805	validation_1-error:0.14279	validation_1-logloss:0.34399
[63]	validation_0-error:0.05368	validation_0-logloss:0.16706	validation_1-error:0.14327	validation_1-logloss:0.34388
[64]	validation_0-error:0.05324	validation_0-logloss:0.16631	validation_1-error:0.14327	validation_1-logloss:0.34391
[65]	validation_0-error:0.05324	validation_0-logloss:0.16600	validation_1-error:0.14230	validation_1-logloss:0.34371
[66]	validation_0-error:0.05339	validation_0-logloss:0.16526	validation_1-error:0.14182	validation_1-logloss:0.34457
[67]	validation_0-error:0.05339	validation_0-logloss:0.16489	validation_1-error:0.14085	validation_1-logloss:0.34427
[68]	validation_0-error:0.05295	validation_0-logloss:0.16410	validation_1-error:0.14036	validation_1-logloss:0.34488
[69]	validation_0-error:0.05295	validation_0-logloss:0.16386	validation_1-error:0.14085	validation_1-logloss:0.34493
[70]	validation_0-error:0.05150	validation_0-logloss:0.16222	validation_1-error:0.14085	validation_1-logloss:0.34565
[71]	validation_0-error:0.04962	validation_0-logloss:0.15950	validation_1-error:0.14570	validation_1-logloss:0.34687
[72]	validation_0-error:0.04889	validation_0-logloss:0.15859	validation_1-error:0.14667	validation_1-logloss:0.34652
[73]	validation_0-error:0.04875	validation_0-logloss:0.15578	validation_1-error:0.14764	validation_1-logloss:0.34540
[74]	validation_0-error:0.04831	validation_0-logloss:0.15546	validation_1-error:0.14716	validation_1-logloss:0.34513
[75]	validation_0-error:0.04773	validation_0-logloss:0.15448	validation_1-error:0.14619	validation_1-logloss:0.34489
[76]	validation_0-error:0.04628	validation_0-logloss:0.15183	validation_1-error:0.14667	validation_1-logloss:0.34515
[77]	validation_0-error:0.04584	validation_0-logloss:0.15130	validation_1-error:0.14764	validation_1-logloss:0.34523
[78]	validation_0-error:0.04584	validation_0-logloss:0.15112	validation_1-error:0.14764	validation_1-logloss:0.34533
[79]	validation_0-error:0.04526	validation_0-logloss:0.15065	validation_1-error:0.14813	validation_1-logloss:0.34551
[80]	validation_0-error:0.04439	validation_0-logloss:0.14958	validation_1-error:0.14862	validation_1-logloss:0.34569
[81]	validation_0-error:0.04222	validation_0-logloss:0.14795	validation_1-error:0.14667	validation_1-logloss:0.34523
[82]	validation_0-error:0.04178	validation_0-logloss:0.14739	validation_1-error:0.14522	validation_1-logloss:0.34500
[83]	validation_0-error:0.04149	validation_0-logloss:0.14636	validation_1-error:0.14570	validation_1-logloss:0.34579
[84]	validation_0-error:0.04048	validation_0-logloss:0.14425	validation_1-error:0.14667	validation_1-logloss:0.34548
[85]	validation_0-error:0.04048	validation_0-logloss:0.14309	validation_1-error:0.14813	validation_1-logloss:0.34587
[86]	validation_0-error:0.04033	validation_0-logloss:0.14265	validation_1-error:0.14667	validation_1-logloss:0.34578
[87]	validation_0-error:0.03946	validation_0-logloss:0.14128	validation_1-error:0.14522	validation_1-logloss:0.34563
[88]	validation_0-error:0.03961	validation_0-logloss:0.14017	validation_1-error:0.14570	validation_1-logloss:0.34659
[89]	validation_0-error:0.03917	validation_0-logloss:0.13976	validation_1-error:0.14619	validation_1-logloss:0.34712
[90]	validation_0-error:0.03903	validation_0-logloss:0.13935	validation_1-error:0.14619	validation_1-logloss:0.34768
[91]	validation_0-error:0.03917	validation_0-logloss:0.13916	validation_1-error:0.14619	validation_1-logloss:0.34762
[92]	validation_0-error:0.03801	validation_0-logloss:0.13857	validation_1-error:0.14619	validation_1-logloss:0.34847
[93]	validation_0-error:0.03743	validation_0-logloss:0.13791	validation_1-error:0.14619	validation_1-logloss:0.34866
[94]	validation_0-error:0.03670	validation_0-logloss:0.13691	validation_1-error:0.14424	validation_1-logloss:0.34859
[95]	validation_0-error:0.03670	validation_0-logloss:0.13600	validation_1-error:0.14570	validation_1-logloss:0.34925
[96]	validation_0-error:0.03612	validation_0-logloss:0.13535	validation_1-error:0.14570	validation_1-logloss:0.34919
[97]	validation_0-error:0.03554	validation_0-logloss:0.13482	validation_1-error:0.14619	validation_1-logloss:0.34908
[98]	validation_0-error:0.03583	validation_0-logloss:0.13406	validation_1-error:0.14570	validation_1-logloss:0.34905
[99]	validation_0-error:0.03569	validation_0-logloss:0.13331	validation_1-error:0.14522	validation_1-logloss:0.34922

Tuning V2

# Define the hyperparameters and their distributions
params = {
    'learning_rate': np.arange(0.01, 0.2, 0.01),
    'max_depth': np.arange(3, 10),
    'subsample': np.arange(0.5, 1.0, 0.1),
    'colsample_bytree': np.arange(0.5, 1.0, 0.1),
    'n_estimators': np.arange(50, 500, 50),
    'gamma': [0, 0.1, 0.2, 0.3],
    'reg_alpha': [0, 0.1, 0.5, 1, 5],
    'reg_lambda': [0, 0.1, 0.5, 1, 5],
}

# Instantiate the XGBClassifier class
xgb_clf = xgb.XGBClassifier(objective='binary:logistic', seed=123)

# Perform a randomized search to find the best hyperparameters
random_search = RandomizedSearchCV(estimator=xgb_clf, param_distributions=params, n_iter=100,
                                   scoring='accuracy', cv=5, n_jobs=-1, random_state=123)
random_search.fit(X_train, Y_train, early_stopping_rounds=10, eval_set=[(X_valid, Y_valid)], verbose=0)

# Print the best hyperparameters and the best score
print("Best hyperparameters: ", random_search.best_params_)
print("Best score: ", random_search.best_score_)

best_params = random_search.best_params_

final_xgb_clf = xgb.XGBClassifier(objective='binary:logistic', seed=123, **best_params)
final_xgb_clf.fit(X_train, Y_train)

# Make predictions onthe test set using the final model
y_pred_final = final_xgb_clf.predict(X_valid)

make_confusion_matrix(confusion_matrix(Y_valid, y_pred_final), group_names=['True Neg','False Pos','False Neg','True Pos'],  categories=['Zero', 'One'], cmap=custom_cmap2)
plot_ROC(Y_valid, y_pred_final)

/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(
/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/xgboost/sklearn.py:835: UserWarning: `early_stopping_rounds` in `fit` method is deprecated for better compatibility with scikit-learn, use `early_stopping_rounds` in constructor or`set_params` instead.
  warnings.warn(

Best hyperparameters:  {'subsample': 0.7999999999999999, 'reg_lambda': 1, 'reg_alpha': 0.1, 'n_estimators': 200, 'max_depth': 5, 'learning_rate': 0.060000000000000005, 'gamma': 0.1, 'colsample_bytree': 0.5}
Best score:  0.8550729320483175

LightGBM

import lightgbm as lgb
from lightgbm import LGBMClassifier
import lightgbm as lgb
from sklearn.model_selection import GridSearchCV

No Tuning

# create the LGBMClassifier model
lgb_model = LGBMClassifier()

# train the LGBMClassifier model using the training data
lgb_model.fit(X_train, Y_train)

# make predictions on the testing data using the trained LGBMClassifier model
y_pred = lgb_model.predict(X_valid)

make_confusion_matrix(confusion_matrix(Y_valid, y_pred), group_names=['True Neg','False Pos','False Neg','True Pos'],  categories=['Zero', 'One'], cmap=custom_cmap2)
plot_ROC(Y_valid, y_pred)

Tuning

# Define the LightGBM model
model = lgb.LGBMClassifier(random_state=42)

# Define the parameter grid for hyperparameter tuning
param_grid = {
    'num_leaves': [31, 63, 127],
    'max_depth': [3, 6, 9],
    'learning_rate': [0.1, 0.01, 0.001],
    'reg_alpha': [0.1, 0.5, 1],
    'reg_lambda': [0.1, 0.5, 1],
    'min_child_samples': [20, 50, 100],
    'subsample': [0.6, 0.8, 1.0],
    'colsample_bytree': [0.6, 0.8, 1.0]
}

# Perform grid search cross-validation to find the best hyperparameters
grid_search = GridSearchCV(estimator=model, param_grid=param_grid, cv=5, scoring='roc_auc', n_jobs=-1)
grid_search.fit(X_train, Y_train)

# Print the best hyperparameters and corresponding AUC score
print("Best hyperparameters:", grid_search.best_params_)

# Evaluate the best model on the validation set
best_model = grid_search.best_estimator_
y_pred = best_model.predict_proba(X_valid)[:, 1]
y_pred = (y_pred > 0.5).astype(int)

make_confusion_matrix(confusion_matrix(Y_valid, y_pred), group_names=['True Neg','False Pos','False Neg','True Pos'],  categories=['Zero', 'One'], cmap=custom_cmap2)
plot_ROC(Y_valid, y_pred)

Best hyperparameters: {'colsample_bytree': 0.6, 'learning_rate': 0.1, 'max_depth': 6, 'min_child_samples': 20, 'num_leaves': 31, 'reg_alpha': 0.1, 'reg_lambda': 0.5, 'subsample': 0.6}

CatBoost

import catboost as cb
from sklearn.model_selection import GridSearchCV
from catboost import CatBoostClassifier

No Tuning

model = cb.CatBoostClassifier(iterations=100, learning_rate=0.1, depth=6)
model.fit(X_train, Y_train)

y_pred = model.predict(X_valid)

make_confusion_matrix(confusion_matrix(Y_valid, y_pred), group_names=['True Neg','False Pos','False Neg','True Pos'],  categories=['Zero', 'One'], cmap=custom_cmap2)
plot_ROC(Y_valid, y_pred)

0:	learn: 0.6364461	total: 58.9ms	remaining: 5.83s
1:	learn: 0.5914606	total: 61.3ms	remaining: 3s
2:	learn: 0.5560489	total: 63.6ms	remaining: 2.06s
3:	learn: 0.5266232	total: 65.9ms	remaining: 1.58s
4:	learn: 0.5065321	total: 67.9ms	remaining: 1.29s
5:	learn: 0.4870657	total: 70ms	remaining: 1.1s
6:	learn: 0.4706666	total: 72.2ms	remaining: 959ms
7:	learn: 0.4586148	total: 74.5ms	remaining: 856ms
8:	learn: 0.4457154	total: 76.6ms	remaining: 774ms
9:	learn: 0.4352360	total: 78.8ms	remaining: 709ms
10:	learn: 0.4262530	total: 80.9ms	remaining: 655ms
11:	learn: 0.4186209	total: 83ms	remaining: 609ms
12:	learn: 0.4114481	total: 85.3ms	remaining: 571ms
13:	learn: 0.4061050	total: 87.3ms	remaining: 536ms
14:	learn: 0.4005093	total: 89.3ms	remaining: 506ms
15:	learn: 0.3958632	total: 91.3ms	remaining: 479ms
16:	learn: 0.3917998	total: 93.4ms	remaining: 456ms
17:	learn: 0.3882721	total: 95.5ms	remaining: 435ms
18:	learn: 0.3855802	total: 97.8ms	remaining: 417ms
19:	learn: 0.3822322	total: 100ms	remaining: 400ms
20:	learn: 0.3780339	total: 103ms	remaining: 388ms
21:	learn: 0.3753511	total: 105ms	remaining: 374ms
22:	learn: 0.3717757	total: 108ms	remaining: 360ms
23:	learn: 0.3693467	total: 110ms	remaining: 349ms
24:	learn: 0.3669562	total: 112ms	remaining: 337ms
25:	learn: 0.3652265	total: 115ms	remaining: 326ms
26:	learn: 0.3636157	total: 117ms	remaining: 315ms
27:	learn: 0.3615791	total: 120ms	remaining: 309ms
28:	learn: 0.3585495	total: 122ms	remaining: 299ms
29:	learn: 0.3567317	total: 124ms	remaining: 290ms
30:	learn: 0.3552127	total: 127ms	remaining: 282ms
31:	learn: 0.3530999	total: 129ms	remaining: 274ms
32:	learn: 0.3511109	total: 131ms	remaining: 266ms
33:	learn: 0.3488738	total: 133ms	remaining: 259ms
34:	learn: 0.3477933	total: 136ms	remaining: 252ms
35:	learn: 0.3459605	total: 138ms	remaining: 246ms
36:	learn: 0.3440381	total: 140ms	remaining: 239ms
37:	learn: 0.3425592	total: 143ms	remaining: 233ms
38:	learn: 0.3408668	total: 145ms	remaining: 227ms
39:	learn: 0.3395437	total: 147ms	remaining: 221ms
40:	learn: 0.3386060	total: 149ms	remaining: 215ms
41:	learn: 0.3374990	total: 152ms	remaining: 210ms
42:	learn: 0.3362582	total: 154ms	remaining: 204ms
43:	learn: 0.3347841	total: 156ms	remaining: 199ms
44:	learn: 0.3335723	total: 158ms	remaining: 194ms
45:	learn: 0.3325320	total: 162ms	remaining: 190ms
46:	learn: 0.3312695	total: 164ms	remaining: 185ms
47:	learn: 0.3301814	total: 167ms	remaining: 181ms
48:	learn: 0.3288178	total: 170ms	remaining: 177ms
49:	learn: 0.3277816	total: 172ms	remaining: 172ms
50:	learn: 0.3270972	total: 174ms	remaining: 167ms
51:	learn: 0.3259377	total: 178ms	remaining: 164ms
52:	learn: 0.3248846	total: 180ms	remaining: 160ms
53:	learn: 0.3237850	total: 183ms	remaining: 156ms
54:	learn: 0.3231821	total: 185ms	remaining: 151ms
55:	learn: 0.3220397	total: 187ms	remaining: 147ms
56:	learn: 0.3213841	total: 189ms	remaining: 143ms
57:	learn: 0.3207362	total: 192ms	remaining: 139ms
58:	learn: 0.3200391	total: 194ms	remaining: 135ms
59:	learn: 0.3194783	total: 196ms	remaining: 131ms
60:	learn: 0.3188326	total: 199ms	remaining: 127ms
61:	learn: 0.3180466	total: 201ms	remaining: 123ms
62:	learn: 0.3168287	total: 203ms	remaining: 119ms
63:	learn: 0.3159501	total: 205ms	remaining: 116ms
64:	learn: 0.3151789	total: 208ms	remaining: 112ms
65:	learn: 0.3142077	total: 210ms	remaining: 108ms
66:	learn: 0.3133875	total: 212ms	remaining: 105ms
67:	learn: 0.3122731	total: 214ms	remaining: 101ms
68:	learn: 0.3118282	total: 217ms	remaining: 97.3ms
69:	learn: 0.3110448	total: 218ms	remaining: 93.6ms
70:	learn: 0.3101288	total: 221ms	remaining: 90.1ms
71:	learn: 0.3093712	total: 223ms	remaining: 86.7ms
72:	learn: 0.3087376	total: 225ms	remaining: 83.3ms
73:	learn: 0.3079221	total: 228ms	remaining: 80ms
74:	learn: 0.3072386	total: 230ms	remaining: 76.6ms
75:	learn: 0.3064039	total: 232ms	remaining: 73.3ms
76:	learn: 0.3057998	total: 234ms	remaining: 70ms
77:	learn: 0.3048694	total: 237ms	remaining: 66.8ms
78:	learn: 0.3044435	total: 239ms	remaining: 63.5ms
79:	learn: 0.3037139	total: 241ms	remaining: 60.3ms
80:	learn: 0.3029886	total: 244ms	remaining: 57.3ms
81:	learn: 0.3021959	total: 246ms	remaining: 54.1ms
82:	learn: 0.3011795	total: 248ms	remaining: 50.9ms
83:	learn: 0.3000693	total: 251ms	remaining: 47.7ms
84:	learn: 0.2994784	total: 253ms	remaining: 44.7ms
85:	learn: 0.2990843	total: 255ms	remaining: 41.6ms
86:	learn: 0.2983653	total: 258ms	remaining: 38.5ms
87:	learn: 0.2977327	total: 260ms	remaining: 35.5ms
88:	learn: 0.2972025	total: 262ms	remaining: 32.4ms
89:	learn: 0.2966094	total: 265ms	remaining: 29.4ms
90:	learn: 0.2961452	total: 267ms	remaining: 26.4ms
91:	learn: 0.2954828	total: 270ms	remaining: 23.5ms
92:	learn: 0.2947394	total: 272ms	remaining: 20.5ms
93:	learn: 0.2939960	total: 275ms	remaining: 17.5ms
94:	learn: 0.2932414	total: 277ms	remaining: 14.6ms
95:	learn: 0.2922218	total: 279ms	remaining: 11.6ms
96:	learn: 0.2912810	total: 282ms	remaining: 8.71ms
97:	learn: 0.2906565	total: 284ms	remaining: 5.8ms
98:	learn: 0.2901387	total: 286ms	remaining: 2.89ms
99:	learn: 0.2893259	total: 288ms	remaining: 0us

Tuning

catboost = CatBoostClassifier(
    iterations=50,
    random_seed=42,
    learning_rate=0.5,
    custom_loss=['AUC', 'Accuracy']
)

catboost.fit(
    X_train, Y_train,
    eval_set=(X_valid, Y_valid),
    verbose=False,
    plot=True
)

y_pred = catboost.predict(X_valid)

make_confusion_matrix(confusion_matrix(Y_valid, y_pred), group_names=['True Neg','False Pos','False Neg','True Pos'],  categories=['Zero', 'One'], cmap=custom_cmap2)
plot_ROC(Y_valid, y_pred)

MetricVisualizer(layout=Layout(align_self='stretch', height='500px'))

Neural Network

import matplotlib.pyplot as plt
%matplotlib inline
import numpy as np
import pandas as pd
import tensorflow as tf
from tensorflow import keras

Tuning

model = keras.models.Sequential()
model.add(keras.layers.Dense(50, input_dim=74, activation='relu'))
model.add(keras.layers.Activation('sigmoid'))
model.add(keras.layers.Dropout(name="drop1", rate=0.2))
model.add(keras.layers.Dense(30, activation='relu'))
model.add(keras.layers.Activation('sigmoid'))
model.add(keras.layers.Dropout(name="drop2", rate=0.2))
model.add(keras.layers.Dense(2, activation='softmax'))

model.summary()
optimizer = keras.optimizers.Adam(learning_rate=0.001)

model.compile(loss = "sparse_categorical_crossentropy", # as y is encoded as a vector of labels and not as a one hot matrix
              optimizer = optimizer,
              metrics = ["accuracy"])

history = model.fit(X_train, Y_train,
                    validation_data=(X_valid, Y_valid), 
                    batch_size = 64,
                    epochs=100)
model.save("checkpoints/dense_nn.h5")

plot_training(history.history)

Model: "sequential_4"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_12 (Dense)            (None, 50)                3750      
                                                                 
 activation (Activation)     (None, 50)                0         
                                                                 
 drop1 (Dropout)             (None, 50)                0         
                                                                 
 dense_13 (Dense)            (None, 30)                1530      
                                                                 
 activation_1 (Activation)   (None, 30)                0         
                                                                 
 drop2 (Dropout)             (None, 30)                0         
                                                                 
 dense_14 (Dense)            (None, 2)                 62        
                                                                 
=================================================================
Total params: 5342 (20.87 KB)
Trainable params: 5342 (20.87 KB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________

WARNING:absl:At this time, the v2.11+ optimizer `tf.keras.optimizers.Adam` runs slowly on M1/M2 Macs, please use the legacy Keras optimizer instead, located at `tf.keras.optimizers.legacy.Adam`.
WARNING:absl:There is a known slowdown when using v2.11+ Keras optimizers on M1/M2 Macs. Falling back to the legacy Keras optimizer, i.e., `tf.keras.optimizers.legacy.Adam`.

Epoch 1/100
108/108 [==============================] - 0s 2ms/step - loss: 0.7060 - accuracy: 0.5495 - val_loss: 0.6614 - val_accuracy: 0.5838
Epoch 2/100
108/108 [==============================] - 0s 1ms/step - loss: 0.6707 - accuracy: 0.5951 - val_loss: 0.5932 - val_accuracy: 0.7232
Epoch 3/100
108/108 [==============================] - 0s 936us/step - loss: 0.6017 - accuracy: 0.6694 - val_loss: 0.5069 - val_accuracy: 0.8164
Epoch 4/100
108/108 [==============================] - 0s 1ms/step - loss: 0.5397 - accuracy: 0.7380 - val_loss: 0.4499 - val_accuracy: 0.8300
Epoch 5/100
108/108 [==============================] - 0s 921us/step - loss: 0.4968 - accuracy: 0.7719 - val_loss: 0.4134 - val_accuracy: 0.8363
Epoch 6/100
108/108 [==============================] - 0s 917us/step - loss: 0.4738 - accuracy: 0.7857 - val_loss: 0.3967 - val_accuracy: 0.8363
Epoch 7/100
108/108 [==============================] - 0s 920us/step - loss: 0.4558 - accuracy: 0.7995 - val_loss: 0.3894 - val_accuracy: 0.8349
Epoch 8/100
108/108 [==============================] - 0s 911us/step - loss: 0.4440 - accuracy: 0.8146 - val_loss: 0.3861 - val_accuracy: 0.8358
Epoch 9/100
108/108 [==============================] - 0s 934us/step - loss: 0.4455 - accuracy: 0.8129 - val_loss: 0.3852 - val_accuracy: 0.8378
Epoch 10/100
108/108 [==============================] - 0s 912us/step - loss: 0.4330 - accuracy: 0.8118 - val_loss: 0.3836 - val_accuracy: 0.8373
Epoch 11/100
108/108 [==============================] - 0s 902us/step - loss: 0.4356 - accuracy: 0.8172 - val_loss: 0.3829 - val_accuracy: 0.8358
Epoch 12/100
108/108 [==============================] - 0s 908us/step - loss: 0.4296 - accuracy: 0.8211 - val_loss: 0.3856 - val_accuracy: 0.8349
Epoch 13/100
108/108 [==============================] - 0s 914us/step - loss: 0.4400 - accuracy: 0.8160 - val_loss: 0.3834 - val_accuracy: 0.8373
Epoch 14/100
108/108 [==============================] - 0s 912us/step - loss: 0.4287 - accuracy: 0.8218 - val_loss: 0.3835 - val_accuracy: 0.8373
Epoch 15/100
108/108 [==============================] - 0s 905us/step - loss: 0.4242 - accuracy: 0.8246 - val_loss: 0.3843 - val_accuracy: 0.8368
Epoch 16/100
108/108 [==============================] - 0s 910us/step - loss: 0.4231 - accuracy: 0.8247 - val_loss: 0.3868 - val_accuracy: 0.8383
Epoch 17/100
108/108 [==============================] - 0s 909us/step - loss: 0.4259 - accuracy: 0.8256 - val_loss: 0.3852 - val_accuracy: 0.8368
Epoch 18/100
108/108 [==============================] - 0s 908us/step - loss: 0.4226 - accuracy: 0.8236 - val_loss: 0.3843 - val_accuracy: 0.8373
Epoch 19/100
108/108 [==============================] - 0s 903us/step - loss: 0.4235 - accuracy: 0.8247 - val_loss: 0.3832 - val_accuracy: 0.8383
Epoch 20/100
108/108 [==============================] - 0s 905us/step - loss: 0.4261 - accuracy: 0.8250 - val_loss: 0.3832 - val_accuracy: 0.8388
Epoch 21/100
108/108 [==============================] - 0s 1ms/step - loss: 0.4210 - accuracy: 0.8265 - val_loss: 0.3831 - val_accuracy: 0.8388
Epoch 22/100
108/108 [==============================] - 0s 906us/step - loss: 0.4155 - accuracy: 0.8262 - val_loss: 0.3834 - val_accuracy: 0.8388
Epoch 23/100
108/108 [==============================] - 0s 910us/step - loss: 0.4149 - accuracy: 0.8301 - val_loss: 0.3841 - val_accuracy: 0.8392
Epoch 24/100
108/108 [==============================] - 0s 917us/step - loss: 0.4213 - accuracy: 0.8246 - val_loss: 0.3840 - val_accuracy: 0.8388
Epoch 25/100
108/108 [==============================] - 0s 910us/step - loss: 0.4207 - accuracy: 0.8268 - val_loss: 0.3841 - val_accuracy: 0.8383
Epoch 26/100
108/108 [==============================] - 0s 963us/step - loss: 0.4204 - accuracy: 0.8300 - val_loss: 0.3839 - val_accuracy: 0.8397
Epoch 27/100
108/108 [==============================] - 0s 1ms/step - loss: 0.4192 - accuracy: 0.8291 - val_loss: 0.3834 - val_accuracy: 0.8373
Epoch 28/100
108/108 [==============================] - 0s 898us/step - loss: 0.4153 - accuracy: 0.8301 - val_loss: 0.3837 - val_accuracy: 0.8358
Epoch 29/100
108/108 [==============================] - 0s 904us/step - loss: 0.4166 - accuracy: 0.8301 - val_loss: 0.3830 - val_accuracy: 0.8388
Epoch 30/100
108/108 [==============================] - 0s 900us/step - loss: 0.4136 - accuracy: 0.8290 - val_loss: 0.3826 - val_accuracy: 0.8383
Epoch 31/100
108/108 [==============================] - 0s 903us/step - loss: 0.4199 - accuracy: 0.8298 - val_loss: 0.3830 - val_accuracy: 0.8373
Epoch 32/100
108/108 [==============================] - 0s 896us/step - loss: 0.4114 - accuracy: 0.8336 - val_loss: 0.3828 - val_accuracy: 0.8378
Epoch 33/100
108/108 [==============================] - 0s 904us/step - loss: 0.4090 - accuracy: 0.8330 - val_loss: 0.3813 - val_accuracy: 0.8402
Epoch 34/100
108/108 [==============================] - 0s 905us/step - loss: 0.4138 - accuracy: 0.8298 - val_loss: 0.3810 - val_accuracy: 0.8422
Epoch 35/100
108/108 [==============================] - 0s 904us/step - loss: 0.4221 - accuracy: 0.8242 - val_loss: 0.3812 - val_accuracy: 0.8407
Epoch 36/100
108/108 [==============================] - 0s 902us/step - loss: 0.4085 - accuracy: 0.8356 - val_loss: 0.3812 - val_accuracy: 0.8378
Epoch 37/100
108/108 [==============================] - 0s 1ms/step - loss: 0.4113 - accuracy: 0.8343 - val_loss: 0.3798 - val_accuracy: 0.8407
Epoch 38/100
108/108 [==============================] - 0s 908us/step - loss: 0.4090 - accuracy: 0.8311 - val_loss: 0.3804 - val_accuracy: 0.8388
Epoch 39/100
108/108 [==============================] - 0s 895us/step - loss: 0.4102 - accuracy: 0.8324 - val_loss: 0.3821 - val_accuracy: 0.8388
Epoch 40/100
108/108 [==============================] - 0s 899us/step - loss: 0.4138 - accuracy: 0.8301 - val_loss: 0.3808 - val_accuracy: 0.8402
Epoch 41/100
108/108 [==============================] - 0s 892us/step - loss: 0.4195 - accuracy: 0.8308 - val_loss: 0.3825 - val_accuracy: 0.8378
Epoch 42/100
108/108 [==============================] - 0s 900us/step - loss: 0.4094 - accuracy: 0.8306 - val_loss: 0.3827 - val_accuracy: 0.8412
Epoch 43/100
108/108 [==============================] - 0s 899us/step - loss: 0.4039 - accuracy: 0.8337 - val_loss: 0.3793 - val_accuracy: 0.8402
Epoch 44/100
108/108 [==============================] - 0s 1ms/step - loss: 0.4104 - accuracy: 0.8349 - val_loss: 0.3799 - val_accuracy: 0.8383
Epoch 45/100
108/108 [==============================] - 0s 908us/step - loss: 0.4084 - accuracy: 0.8336 - val_loss: 0.3795 - val_accuracy: 0.8392
Epoch 46/100
108/108 [==============================] - 0s 903us/step - loss: 0.4033 - accuracy: 0.8364 - val_loss: 0.3797 - val_accuracy: 0.8397
Epoch 47/100
108/108 [==============================] - 0s 902us/step - loss: 0.4111 - accuracy: 0.8323 - val_loss: 0.3798 - val_accuracy: 0.8388
Epoch 48/100
108/108 [==============================] - 0s 901us/step - loss: 0.4129 - accuracy: 0.8317 - val_loss: 0.3809 - val_accuracy: 0.8392
Epoch 49/100
108/108 [==============================] - 0s 900us/step - loss: 0.4053 - accuracy: 0.8298 - val_loss: 0.3798 - val_accuracy: 0.8378
Epoch 50/100
108/108 [==============================] - 0s 906us/step - loss: 0.4088 - accuracy: 0.8339 - val_loss: 0.3781 - val_accuracy: 0.8407
Epoch 51/100
108/108 [==============================] - 0s 952us/step - loss: 0.4038 - accuracy: 0.8337 - val_loss: 0.3795 - val_accuracy: 0.8436
Epoch 52/100
108/108 [==============================] - 0s 1ms/step - loss: 0.4080 - accuracy: 0.8336 - val_loss: 0.3799 - val_accuracy: 0.8388
Epoch 53/100
108/108 [==============================] - 0s 914us/step - loss: 0.4009 - accuracy: 0.8337 - val_loss: 0.3793 - val_accuracy: 0.8402
Epoch 54/100
108/108 [==============================] - 0s 930us/step - loss: 0.4013 - accuracy: 0.8353 - val_loss: 0.3768 - val_accuracy: 0.8436
Epoch 55/100
108/108 [==============================] - 0s 909us/step - loss: 0.4034 - accuracy: 0.8356 - val_loss: 0.3782 - val_accuracy: 0.8397
Epoch 56/100
108/108 [==============================] - 0s 907us/step - loss: 0.4073 - accuracy: 0.8324 - val_loss: 0.3764 - val_accuracy: 0.8397
Epoch 57/100
108/108 [==============================] - 0s 907us/step - loss: 0.4000 - accuracy: 0.8353 - val_loss: 0.3772 - val_accuracy: 0.8407
Epoch 58/100
108/108 [==============================] - 0s 904us/step - loss: 0.4028 - accuracy: 0.8349 - val_loss: 0.3761 - val_accuracy: 0.8417
Epoch 59/100
108/108 [==============================] - 0s 904us/step - loss: 0.4116 - accuracy: 0.8308 - val_loss: 0.3751 - val_accuracy: 0.8456
Epoch 60/100
108/108 [==============================] - 0s 906us/step - loss: 0.4049 - accuracy: 0.8332 - val_loss: 0.3751 - val_accuracy: 0.8446
Epoch 61/100
108/108 [==============================] - 0s 899us/step - loss: 0.4093 - accuracy: 0.8339 - val_loss: 0.3753 - val_accuracy: 0.8441
Epoch 62/100
108/108 [==============================] - 0s 907us/step - loss: 0.4069 - accuracy: 0.8375 - val_loss: 0.3745 - val_accuracy: 0.8436
Epoch 63/100
108/108 [==============================] - 0s 909us/step - loss: 0.4027 - accuracy: 0.8365 - val_loss: 0.3744 - val_accuracy: 0.8431
Epoch 64/100
108/108 [==============================] - 0s 903us/step - loss: 0.3958 - accuracy: 0.8374 - val_loss: 0.3752 - val_accuracy: 0.8456
Epoch 65/100
108/108 [==============================] - 0s 897us/step - loss: 0.4040 - accuracy: 0.8351 - val_loss: 0.3733 - val_accuracy: 0.8470
Epoch 66/100
108/108 [==============================] - 0s 896us/step - loss: 0.3986 - accuracy: 0.8372 - val_loss: 0.3734 - val_accuracy: 0.8456
Epoch 67/100
108/108 [==============================] - 0s 896us/step - loss: 0.4038 - accuracy: 0.8342 - val_loss: 0.3740 - val_accuracy: 0.8456
Epoch 68/100
108/108 [==============================] - 0s 908us/step - loss: 0.4044 - accuracy: 0.8345 - val_loss: 0.3751 - val_accuracy: 0.8407
Epoch 69/100
108/108 [==============================] - 0s 898us/step - loss: 0.3996 - accuracy: 0.8369 - val_loss: 0.3741 - val_accuracy: 0.8456
Epoch 70/100
108/108 [==============================] - 0s 900us/step - loss: 0.3977 - accuracy: 0.8375 - val_loss: 0.3722 - val_accuracy: 0.8460
Epoch 71/100
108/108 [==============================] - 0s 907us/step - loss: 0.4062 - accuracy: 0.8324 - val_loss: 0.3724 - val_accuracy: 0.8465
Epoch 72/100
108/108 [==============================] - 0s 902us/step - loss: 0.4018 - accuracy: 0.8346 - val_loss: 0.3741 - val_accuracy: 0.8417
Epoch 73/100
108/108 [==============================] - 0s 1ms/step - loss: 0.4042 - accuracy: 0.8330 - val_loss: 0.3740 - val_accuracy: 0.8412
Epoch 74/100
108/108 [==============================] - 0s 910us/step - loss: 0.3955 - accuracy: 0.8346 - val_loss: 0.3715 - val_accuracy: 0.8475
Epoch 75/100
108/108 [==============================] - 0s 1ms/step - loss: 0.3974 - accuracy: 0.8387 - val_loss: 0.3721 - val_accuracy: 0.8456
Epoch 76/100
108/108 [==============================] - 0s 935us/step - loss: 0.3966 - accuracy: 0.8407 - val_loss: 0.3719 - val_accuracy: 0.8451
Epoch 77/100
108/108 [==============================] - 0s 903us/step - loss: 0.3935 - accuracy: 0.8388 - val_loss: 0.3744 - val_accuracy: 0.8412
Epoch 78/100
108/108 [==============================] - 0s 915us/step - loss: 0.4039 - accuracy: 0.8346 - val_loss: 0.3751 - val_accuracy: 0.8397
Epoch 79/100
108/108 [==============================] - 0s 908us/step - loss: 0.3972 - accuracy: 0.8395 - val_loss: 0.3726 - val_accuracy: 0.8412
Epoch 80/100
108/108 [==============================] - 0s 910us/step - loss: 0.3946 - accuracy: 0.8356 - val_loss: 0.3711 - val_accuracy: 0.8456
Epoch 81/100
108/108 [==============================] - 0s 909us/step - loss: 0.4000 - accuracy: 0.8339 - val_loss: 0.3702 - val_accuracy: 0.8485
Epoch 82/100
108/108 [==============================] - 0s 908us/step - loss: 0.3999 - accuracy: 0.8340 - val_loss: 0.3716 - val_accuracy: 0.8417
Epoch 83/100
108/108 [==============================] - 0s 905us/step - loss: 0.3959 - accuracy: 0.8388 - val_loss: 0.3704 - val_accuracy: 0.8475
Epoch 84/100
108/108 [==============================] - 0s 908us/step - loss: 0.3897 - accuracy: 0.8375 - val_loss: 0.3727 - val_accuracy: 0.8422
Epoch 85/100
108/108 [==============================] - 0s 905us/step - loss: 0.4006 - accuracy: 0.8403 - val_loss: 0.3706 - val_accuracy: 0.8465
Epoch 86/100
108/108 [==============================] - 0s 905us/step - loss: 0.3965 - accuracy: 0.8369 - val_loss: 0.3722 - val_accuracy: 0.8475
Epoch 87/100
108/108 [==============================] - 0s 904us/step - loss: 0.3951 - accuracy: 0.8380 - val_loss: 0.3708 - val_accuracy: 0.8470
Epoch 88/100
108/108 [==============================] - 0s 899us/step - loss: 0.3941 - accuracy: 0.8380 - val_loss: 0.3718 - val_accuracy: 0.8436
Epoch 89/100
108/108 [==============================] - 0s 913us/step - loss: 0.3953 - accuracy: 0.8358 - val_loss: 0.3696 - val_accuracy: 0.8470
Epoch 90/100
108/108 [==============================] - 0s 915us/step - loss: 0.3941 - accuracy: 0.8342 - val_loss: 0.3701 - val_accuracy: 0.8470
Epoch 91/100
108/108 [==============================] - 0s 902us/step - loss: 0.4039 - accuracy: 0.8307 - val_loss: 0.3688 - val_accuracy: 0.8475
Epoch 92/100
108/108 [==============================] - 0s 906us/step - loss: 0.3962 - accuracy: 0.8366 - val_loss: 0.3690 - val_accuracy: 0.8490
Epoch 93/100
108/108 [==============================] - 0s 899us/step - loss: 0.3943 - accuracy: 0.8351 - val_loss: 0.3689 - val_accuracy: 0.8460
Epoch 94/100
108/108 [==============================] - 0s 901us/step - loss: 0.3884 - accuracy: 0.8388 - val_loss: 0.3683 - val_accuracy: 0.8475
Epoch 95/100
108/108 [==============================] - 0s 896us/step - loss: 0.3910 - accuracy: 0.8414 - val_loss: 0.3690 - val_accuracy: 0.8475
Epoch 96/100
108/108 [==============================] - 0s 901us/step - loss: 0.3980 - accuracy: 0.8326 - val_loss: 0.3704 - val_accuracy: 0.8465
Epoch 97/100
108/108 [==============================] - 0s 1ms/step - loss: 0.3879 - accuracy: 0.8424 - val_loss: 0.3687 - val_accuracy: 0.8475
Epoch 98/100
108/108 [==============================] - 0s 902us/step - loss: 0.3947 - accuracy: 0.8380 - val_loss: 0.3686 - val_accuracy: 0.8470
Epoch 99/100
108/108 [==============================] - 0s 905us/step - loss: 0.3973 - accuracy: 0.8330 - val_loss: 0.3679 - val_accuracy: 0.8470
Epoch 100/100
108/108 [==============================] - 0s 898us/step - loss: 0.3920 - accuracy: 0.8398 - val_loss: 0.3682 - val_accuracy: 0.8460

/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/keras/src/engine/training.py:3000: UserWarning: You are saving your model as an HDF5 file via `model.save()`. This file format is considered legacy. We recommend using instead the native Keras format, e.g. `model.save('my_model.keras')`.
  saving_api.save_model(

model = keras.models.Sequential()
model.add(keras.layers.Dense(50, input_dim=74, activation='relu'))
model.add(keras.layers.Dense(30, activation='relu'))
model.add(keras.layers.Dropout(name="drop2", rate=0.2))
model.add(keras.layers.Dense(2, activation='softmax'))

model.summary()
optimizer = keras.optimizers.Adam(learning_rate=0.001)

model.compile(loss = "sparse_categorical_crossentropy", # as y is encoded as a vector of labels and not as a one hot matrix
              optimizer = optimizer,
              metrics = ["accuracy"])

history = model.fit(X_train, Y_train,
                    validation_data=(X_valid, Y_valid), 
                    batch_size = 64,
                    epochs=100)
model.save("checkpoints/dense_nn2.h5")

plot_training(history.history)

Model: "sequential_5"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_15 (Dense)            (None, 50)                3750      
                                                                 
 dense_16 (Dense)            (None, 30)                1530      
                                                                 
 drop2 (Dropout)             (None, 30)                0         
                                                                 
 dense_17 (Dense)            (None, 2)                 62        
                                                                 
=================================================================
Total params: 5342 (20.87 KB)
Trainable params: 5342 (20.87 KB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________

WARNING:absl:At this time, the v2.11+ optimizer `tf.keras.optimizers.Adam` runs slowly on M1/M2 Macs, please use the legacy Keras optimizer instead, located at `tf.keras.optimizers.legacy.Adam`.
WARNING:absl:There is a known slowdown when using v2.11+ Keras optimizers on M1/M2 Macs. Falling back to the legacy Keras optimizer, i.e., `tf.keras.optimizers.legacy.Adam`.

Epoch 1/100
108/108 [==============================] - 0s 2ms/step - loss: 0.5448 - accuracy: 0.7312 - val_loss: 0.4179 - val_accuracy: 0.8271
Epoch 2/100
108/108 [==============================] - 0s 914us/step - loss: 0.3937 - accuracy: 0.8298 - val_loss: 0.3833 - val_accuracy: 0.8388
Epoch 3/100
108/108 [==============================] - 0s 897us/step - loss: 0.3626 - accuracy: 0.8477 - val_loss: 0.3709 - val_accuracy: 0.8456
Epoch 4/100
108/108 [==============================] - 0s 895us/step - loss: 0.3505 - accuracy: 0.8509 - val_loss: 0.3677 - val_accuracy: 0.8475
Epoch 5/100
108/108 [==============================] - 0s 886us/step - loss: 0.3413 - accuracy: 0.8571 - val_loss: 0.3665 - val_accuracy: 0.8460
Epoch 6/100
108/108 [==============================] - 0s 909us/step - loss: 0.3296 - accuracy: 0.8601 - val_loss: 0.3593 - val_accuracy: 0.8509
Epoch 7/100
108/108 [==============================] - 0s 918us/step - loss: 0.3217 - accuracy: 0.8629 - val_loss: 0.3574 - val_accuracy: 0.8494
Epoch 8/100
108/108 [==============================] - 0s 920us/step - loss: 0.3163 - accuracy: 0.8638 - val_loss: 0.3527 - val_accuracy: 0.8553
Epoch 9/100
108/108 [==============================] - 0s 898us/step - loss: 0.3116 - accuracy: 0.8713 - val_loss: 0.3505 - val_accuracy: 0.8470
Epoch 10/100
108/108 [==============================] - 0s 883us/step - loss: 0.2999 - accuracy: 0.8738 - val_loss: 0.3550 - val_accuracy: 0.8528
Epoch 11/100
108/108 [==============================] - 0s 897us/step - loss: 0.2964 - accuracy: 0.8726 - val_loss: 0.3519 - val_accuracy: 0.8504
Epoch 12/100
108/108 [==============================] - 0s 886us/step - loss: 0.2900 - accuracy: 0.8777 - val_loss: 0.3509 - val_accuracy: 0.8480
Epoch 13/100
108/108 [==============================] - 0s 886us/step - loss: 0.2809 - accuracy: 0.8828 - val_loss: 0.3497 - val_accuracy: 0.8553
Epoch 14/100
108/108 [==============================] - 0s 886us/step - loss: 0.2697 - accuracy: 0.8893 - val_loss: 0.3571 - val_accuracy: 0.8499
Epoch 15/100
108/108 [==============================] - 0s 881us/step - loss: 0.2689 - accuracy: 0.8887 - val_loss: 0.3551 - val_accuracy: 0.8504
Epoch 16/100
108/108 [==============================] - 0s 894us/step - loss: 0.2652 - accuracy: 0.8905 - val_loss: 0.3565 - val_accuracy: 0.8524
Epoch 17/100
108/108 [==============================] - 0s 889us/step - loss: 0.2615 - accuracy: 0.8912 - val_loss: 0.3546 - val_accuracy: 0.8543
Epoch 18/100
108/108 [==============================] - 0s 925us/step - loss: 0.2537 - accuracy: 0.8937 - val_loss: 0.3572 - val_accuracy: 0.8499
Epoch 19/100
108/108 [==============================] - 0s 907us/step - loss: 0.2498 - accuracy: 0.8974 - val_loss: 0.3587 - val_accuracy: 0.8562
Epoch 20/100
108/108 [==============================] - 0s 915us/step - loss: 0.2451 - accuracy: 0.8966 - val_loss: 0.3652 - val_accuracy: 0.8494
Epoch 21/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2402 - accuracy: 0.9022 - val_loss: 0.3655 - val_accuracy: 0.8470
Epoch 22/100
108/108 [==============================] - 0s 905us/step - loss: 0.2379 - accuracy: 0.9029 - val_loss: 0.3677 - val_accuracy: 0.8538
Epoch 23/100
108/108 [==============================] - 0s 893us/step - loss: 0.2273 - accuracy: 0.9069 - val_loss: 0.3760 - val_accuracy: 0.8524
Epoch 24/100
108/108 [==============================] - 0s 902us/step - loss: 0.2266 - accuracy: 0.9067 - val_loss: 0.3780 - val_accuracy: 0.8470
Epoch 25/100
108/108 [==============================] - 0s 889us/step - loss: 0.2235 - accuracy: 0.9077 - val_loss: 0.3752 - val_accuracy: 0.8543
Epoch 26/100
108/108 [==============================] - 0s 885us/step - loss: 0.2159 - accuracy: 0.9132 - val_loss: 0.3855 - val_accuracy: 0.8499
Epoch 27/100
108/108 [==============================] - 0s 888us/step - loss: 0.2120 - accuracy: 0.9125 - val_loss: 0.3794 - val_accuracy: 0.8519
Epoch 28/100
108/108 [==============================] - 0s 896us/step - loss: 0.2095 - accuracy: 0.9128 - val_loss: 0.3951 - val_accuracy: 0.8485
Epoch 29/100
108/108 [==============================] - 0s 905us/step - loss: 0.1995 - accuracy: 0.9185 - val_loss: 0.4036 - val_accuracy: 0.8460
Epoch 30/100
108/108 [==============================] - 0s 890us/step - loss: 0.2034 - accuracy: 0.9144 - val_loss: 0.3905 - val_accuracy: 0.8567
Epoch 31/100
108/108 [==============================] - 0s 857us/step - loss: 0.1957 - accuracy: 0.9183 - val_loss: 0.4155 - val_accuracy: 0.8494
Epoch 32/100
108/108 [==============================] - 0s 850us/step - loss: 0.1937 - accuracy: 0.9222 - val_loss: 0.4180 - val_accuracy: 0.8470
Epoch 33/100
108/108 [==============================] - 0s 856us/step - loss: 0.1908 - accuracy: 0.9230 - val_loss: 0.4143 - val_accuracy: 0.8504
Epoch 34/100
108/108 [==============================] - 0s 901us/step - loss: 0.1868 - accuracy: 0.9222 - val_loss: 0.4152 - val_accuracy: 0.8504
Epoch 35/100
108/108 [==============================] - 0s 900us/step - loss: 0.1838 - accuracy: 0.9264 - val_loss: 0.4255 - val_accuracy: 0.8494
Epoch 36/100
108/108 [==============================] - 0s 935us/step - loss: 0.1800 - accuracy: 0.9264 - val_loss: 0.4310 - val_accuracy: 0.8490
Epoch 37/100
108/108 [==============================] - 0s 890us/step - loss: 0.1764 - accuracy: 0.9280 - val_loss: 0.4269 - val_accuracy: 0.8509
Epoch 38/100
108/108 [==============================] - 0s 881us/step - loss: 0.1725 - accuracy: 0.9305 - val_loss: 0.4486 - val_accuracy: 0.8475
Epoch 39/100
108/108 [==============================] - 0s 875us/step - loss: 0.1719 - accuracy: 0.9302 - val_loss: 0.4608 - val_accuracy: 0.8480
Epoch 40/100
108/108 [==============================] - 0s 853us/step - loss: 0.1657 - accuracy: 0.9336 - val_loss: 0.4438 - val_accuracy: 0.8490
Epoch 41/100
108/108 [==============================] - 0s 853us/step - loss: 0.1630 - accuracy: 0.9363 - val_loss: 0.4572 - val_accuracy: 0.8426
Epoch 42/100
108/108 [==============================] - 0s 852us/step - loss: 0.1613 - accuracy: 0.9347 - val_loss: 0.4669 - val_accuracy: 0.8499
Epoch 43/100
108/108 [==============================] - 0s 848us/step - loss: 0.1567 - accuracy: 0.9372 - val_loss: 0.4850 - val_accuracy: 0.8480
Epoch 44/100
108/108 [==============================] - 0s 1ms/step - loss: 0.1561 - accuracy: 0.9370 - val_loss: 0.4524 - val_accuracy: 0.8407
Epoch 45/100
108/108 [==============================] - 0s 856us/step - loss: 0.1510 - accuracy: 0.9391 - val_loss: 0.4968 - val_accuracy: 0.8436
Epoch 46/100
108/108 [==============================] - 0s 846us/step - loss: 0.1495 - accuracy: 0.9369 - val_loss: 0.4871 - val_accuracy: 0.8407
Epoch 47/100
108/108 [==============================] - 0s 849us/step - loss: 0.1451 - accuracy: 0.9405 - val_loss: 0.5054 - val_accuracy: 0.8456
Epoch 48/100
108/108 [==============================] - 0s 850us/step - loss: 0.1445 - accuracy: 0.9427 - val_loss: 0.5089 - val_accuracy: 0.8373
Epoch 49/100
108/108 [==============================] - 0s 853us/step - loss: 0.1387 - accuracy: 0.9466 - val_loss: 0.5133 - val_accuracy: 0.8446
Epoch 50/100
108/108 [==============================] - 0s 855us/step - loss: 0.1370 - accuracy: 0.9485 - val_loss: 0.5091 - val_accuracy: 0.8426
Epoch 51/100
108/108 [==============================] - 0s 852us/step - loss: 0.1360 - accuracy: 0.9437 - val_loss: 0.5403 - val_accuracy: 0.8363
Epoch 52/100
108/108 [==============================] - 0s 908us/step - loss: 0.1302 - accuracy: 0.9508 - val_loss: 0.5599 - val_accuracy: 0.8397
Epoch 53/100
108/108 [==============================] - 0s 923us/step - loss: 0.1303 - accuracy: 0.9463 - val_loss: 0.5415 - val_accuracy: 0.8426
Epoch 54/100
108/108 [==============================] - 0s 901us/step - loss: 0.1262 - accuracy: 0.9504 - val_loss: 0.5537 - val_accuracy: 0.8378
Epoch 55/100
108/108 [==============================] - 0s 901us/step - loss: 0.1253 - accuracy: 0.9511 - val_loss: 0.5613 - val_accuracy: 0.8378
Epoch 56/100
108/108 [==============================] - 0s 959us/step - loss: 0.1303 - accuracy: 0.9492 - val_loss: 0.5505 - val_accuracy: 0.8407
Epoch 57/100
108/108 [==============================] - 0s 910us/step - loss: 0.1194 - accuracy: 0.9547 - val_loss: 0.5757 - val_accuracy: 0.8349
Epoch 58/100
108/108 [==============================] - 0s 924us/step - loss: 0.1162 - accuracy: 0.9546 - val_loss: 0.5841 - val_accuracy: 0.8358
Epoch 59/100
108/108 [==============================] - 0s 1ms/step - loss: 0.1188 - accuracy: 0.9498 - val_loss: 0.5879 - val_accuracy: 0.8339
Epoch 60/100
108/108 [==============================] - 0s 891us/step - loss: 0.1124 - accuracy: 0.9549 - val_loss: 0.5974 - val_accuracy: 0.8417
Epoch 61/100
108/108 [==============================] - 0s 890us/step - loss: 0.1118 - accuracy: 0.9562 - val_loss: 0.6184 - val_accuracy: 0.8378
Epoch 62/100
108/108 [==============================] - 0s 886us/step - loss: 0.1154 - accuracy: 0.9539 - val_loss: 0.5951 - val_accuracy: 0.8392
Epoch 63/100
108/108 [==============================] - 0s 884us/step - loss: 0.1058 - accuracy: 0.9588 - val_loss: 0.6298 - val_accuracy: 0.8392
Epoch 64/100
108/108 [==============================] - 0s 1ms/step - loss: 0.1098 - accuracy: 0.9563 - val_loss: 0.6273 - val_accuracy: 0.8349
Epoch 65/100
108/108 [==============================] - 0s 924us/step - loss: 0.1019 - accuracy: 0.9595 - val_loss: 0.6511 - val_accuracy: 0.8358
Epoch 66/100
108/108 [==============================] - 0s 914us/step - loss: 0.1027 - accuracy: 0.9578 - val_loss: 0.6489 - val_accuracy: 0.8378
Epoch 67/100
108/108 [==============================] - 0s 925us/step - loss: 0.1030 - accuracy: 0.9591 - val_loss: 0.6569 - val_accuracy: 0.8349
Epoch 68/100
108/108 [==============================] - 0s 923us/step - loss: 0.1029 - accuracy: 0.9602 - val_loss: 0.6700 - val_accuracy: 0.8339
Epoch 69/100
108/108 [==============================] - 0s 939us/step - loss: 0.0961 - accuracy: 0.9624 - val_loss: 0.6816 - val_accuracy: 0.8363
Epoch 70/100
108/108 [==============================] - 0s 915us/step - loss: 0.0957 - accuracy: 0.9636 - val_loss: 0.6728 - val_accuracy: 0.8358
Epoch 71/100
108/108 [==============================] - 0s 1ms/step - loss: 0.0960 - accuracy: 0.9627 - val_loss: 0.7028 - val_accuracy: 0.8383
Epoch 72/100
108/108 [==============================] - 0s 945us/step - loss: 0.0931 - accuracy: 0.9640 - val_loss: 0.7079 - val_accuracy: 0.8329
Epoch 73/100
108/108 [==============================] - 0s 954us/step - loss: 0.0939 - accuracy: 0.9656 - val_loss: 0.7127 - val_accuracy: 0.8290
Epoch 74/100
108/108 [==============================] - 0s 913us/step - loss: 0.0911 - accuracy: 0.9647 - val_loss: 0.7142 - val_accuracy: 0.8329
Epoch 75/100
108/108 [==============================] - 0s 890us/step - loss: 0.0913 - accuracy: 0.9629 - val_loss: 0.7449 - val_accuracy: 0.8354
Epoch 76/100
108/108 [==============================] - 0s 904us/step - loss: 0.0922 - accuracy: 0.9623 - val_loss: 0.7369 - val_accuracy: 0.8286
Epoch 77/100
108/108 [==============================] - 0s 899us/step - loss: 0.0869 - accuracy: 0.9678 - val_loss: 0.7539 - val_accuracy: 0.8354
Epoch 78/100
108/108 [==============================] - 0s 881us/step - loss: 0.0814 - accuracy: 0.9704 - val_loss: 0.7708 - val_accuracy: 0.8276
Epoch 79/100
108/108 [==============================] - 0s 877us/step - loss: 0.0838 - accuracy: 0.9679 - val_loss: 0.7844 - val_accuracy: 0.8324
Epoch 80/100
108/108 [==============================] - 0s 875us/step - loss: 0.0853 - accuracy: 0.9663 - val_loss: 0.7823 - val_accuracy: 0.8310
Epoch 81/100
108/108 [==============================] - 0s 871us/step - loss: 0.0780 - accuracy: 0.9703 - val_loss: 0.8222 - val_accuracy: 0.8300
Epoch 82/100
108/108 [==============================] - 0s 870us/step - loss: 0.0773 - accuracy: 0.9708 - val_loss: 0.8171 - val_accuracy: 0.8295
Epoch 83/100
108/108 [==============================] - 0s 1ms/step - loss: 0.0757 - accuracy: 0.9708 - val_loss: 0.8280 - val_accuracy: 0.8276
Epoch 84/100
108/108 [==============================] - 0s 881us/step - loss: 0.0772 - accuracy: 0.9711 - val_loss: 0.8294 - val_accuracy: 0.8271
Epoch 85/100
108/108 [==============================] - 0s 1ms/step - loss: 0.0739 - accuracy: 0.9717 - val_loss: 0.8783 - val_accuracy: 0.8208
Epoch 86/100
108/108 [==============================] - 0s 878us/step - loss: 0.0819 - accuracy: 0.9687 - val_loss: 0.8720 - val_accuracy: 0.8242
Epoch 87/100
108/108 [==============================] - 0s 869us/step - loss: 0.0741 - accuracy: 0.9719 - val_loss: 0.8532 - val_accuracy: 0.8252
Epoch 88/100
108/108 [==============================] - 0s 863us/step - loss: 0.0759 - accuracy: 0.9717 - val_loss: 0.8852 - val_accuracy: 0.8281
Epoch 89/100
108/108 [==============================] - 0s 878us/step - loss: 0.0830 - accuracy: 0.9659 - val_loss: 0.8567 - val_accuracy: 0.8208
Epoch 90/100
108/108 [==============================] - 0s 873us/step - loss: 0.0767 - accuracy: 0.9704 - val_loss: 0.8988 - val_accuracy: 0.8349
Epoch 91/100
108/108 [==============================] - 0s 872us/step - loss: 0.0673 - accuracy: 0.9742 - val_loss: 0.8959 - val_accuracy: 0.8358
Epoch 92/100
108/108 [==============================] - 0s 868us/step - loss: 0.0674 - accuracy: 0.9759 - val_loss: 0.9126 - val_accuracy: 0.8271
Epoch 93/100
108/108 [==============================] - 0s 863us/step - loss: 0.0661 - accuracy: 0.9768 - val_loss: 0.9440 - val_accuracy: 0.8261
Epoch 94/100
108/108 [==============================] - 0s 868us/step - loss: 0.0685 - accuracy: 0.9745 - val_loss: 0.9448 - val_accuracy: 0.8252
Epoch 95/100
108/108 [==============================] - 0s 878us/step - loss: 0.0613 - accuracy: 0.9784 - val_loss: 0.9899 - val_accuracy: 0.8256
Epoch 96/100
108/108 [==============================] - 0s 870us/step - loss: 0.0701 - accuracy: 0.9746 - val_loss: 0.9558 - val_accuracy: 0.8242
Epoch 97/100
108/108 [==============================] - 0s 1ms/step - loss: 0.0607 - accuracy: 0.9798 - val_loss: 0.9781 - val_accuracy: 0.8242
Epoch 98/100
108/108 [==============================] - 0s 900us/step - loss: 0.0642 - accuracy: 0.9768 - val_loss: 0.9919 - val_accuracy: 0.8218
Epoch 99/100
108/108 [==============================] - 0s 873us/step - loss: 0.0675 - accuracy: 0.9768 - val_loss: 1.0310 - val_accuracy: 0.8334
Epoch 100/100
108/108 [==============================] - 0s 870us/step - loss: 0.0686 - accuracy: 0.9750 - val_loss: 0.9790 - val_accuracy: 0.8252

/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/keras/src/engine/training.py:3000: UserWarning: You are saving your model as an HDF5 file via `model.save()`. This file format is considered legacy. We recommend using instead the native Keras format, e.g. `model.save('my_model.keras')`.
  saving_api.save_model(

model = keras.models.Sequential()
model.add(keras.layers.Dense(300, input_dim=74, activation='relu'))
model.add(keras.layers.Dropout(name="drop2", rate=0.2))
model.add(keras.layers.Dense(100, activation='relu'))
model.add(keras.layers.Dropout(name="drop3", rate=0.2))
model.add(keras.layers.Dense(2, activation='softmax'))

model.summary()
optimizer = keras.optimizers.Adam(learning_rate=0.0001) # SGD, Nadam, 

model.compile(loss = "sparse_categorical_crossentropy", # as y is encoded as a vector of labels and not as a one hot matrix
              optimizer = optimizer,
              metrics = ["accuracy"])

history = model.fit(X_train, Y_train,
                    validation_data=(X_valid, Y_valid), 
                    batch_size = 64,
                    epochs=100)
model.save("checkpoints/dense_nn3.h5")

plot_training(history.history)

Model: "sequential_6"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense_18 (Dense)            (None, 300)               22500     
                                                                 
 drop2 (Dropout)             (None, 300)               0         
                                                                 
 dense_19 (Dense)            (None, 100)               30100     
                                                                 
 drop3 (Dropout)             (None, 100)               0         
                                                                 
 dense_20 (Dense)            (None, 2)                 202       
                                                                 
=================================================================
Total params: 52802 (206.26 KB)
Trainable params: 52802 (206.26 KB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________

WARNING:absl:At this time, the v2.11+ optimizer `tf.keras.optimizers.Adam` runs slowly on M1/M2 Macs, please use the legacy Keras optimizer instead, located at `tf.keras.optimizers.legacy.Adam`.
WARNING:absl:There is a known slowdown when using v2.11+ Keras optimizers on M1/M2 Macs. Falling back to the legacy Keras optimizer, i.e., `tf.keras.optimizers.legacy.Adam`.

Epoch 1/100
108/108 [==============================] - 1s 3ms/step - loss: 0.6182 - accuracy: 0.6784 - val_loss: 0.5328 - val_accuracy: 0.7640
Epoch 2/100
108/108 [==============================] - 0s 1ms/step - loss: 0.4911 - accuracy: 0.7812 - val_loss: 0.4422 - val_accuracy: 0.8048
Epoch 3/100
108/108 [==============================] - 0s 1ms/step - loss: 0.4302 - accuracy: 0.8139 - val_loss: 0.4090 - val_accuracy: 0.8256
Epoch 4/100
108/108 [==============================] - 0s 1ms/step - loss: 0.4013 - accuracy: 0.8246 - val_loss: 0.3951 - val_accuracy: 0.8349
Epoch 5/100
108/108 [==============================] - 0s 1ms/step - loss: 0.3833 - accuracy: 0.8330 - val_loss: 0.3861 - val_accuracy: 0.8378
Epoch 6/100
108/108 [==============================] - 0s 1ms/step - loss: 0.3736 - accuracy: 0.8390 - val_loss: 0.3803 - val_accuracy: 0.8412
Epoch 7/100
108/108 [==============================] - 0s 1ms/step - loss: 0.3663 - accuracy: 0.8404 - val_loss: 0.3751 - val_accuracy: 0.8451
Epoch 8/100
108/108 [==============================] - 0s 1ms/step - loss: 0.3610 - accuracy: 0.8500 - val_loss: 0.3724 - val_accuracy: 0.8426
Epoch 9/100
108/108 [==============================] - 0s 1ms/step - loss: 0.3545 - accuracy: 0.8475 - val_loss: 0.3672 - val_accuracy: 0.8456
Epoch 10/100
108/108 [==============================] - 0s 1ms/step - loss: 0.3544 - accuracy: 0.8465 - val_loss: 0.3641 - val_accuracy: 0.8494
Epoch 11/100
108/108 [==============================] - 0s 1ms/step - loss: 0.3448 - accuracy: 0.8522 - val_loss: 0.3627 - val_accuracy: 0.8465
Epoch 12/100
108/108 [==============================] - 0s 1ms/step - loss: 0.3473 - accuracy: 0.8504 - val_loss: 0.3602 - val_accuracy: 0.8509
Epoch 13/100
108/108 [==============================] - 0s 1ms/step - loss: 0.3420 - accuracy: 0.8554 - val_loss: 0.3582 - val_accuracy: 0.8475
Epoch 14/100
108/108 [==============================] - 0s 2ms/step - loss: 0.3406 - accuracy: 0.8510 - val_loss: 0.3565 - val_accuracy: 0.8509
Epoch 15/100
108/108 [==============================] - 0s 2ms/step - loss: 0.3358 - accuracy: 0.8554 - val_loss: 0.3550 - val_accuracy: 0.8509
Epoch 16/100
108/108 [==============================] - 0s 1ms/step - loss: 0.3334 - accuracy: 0.8558 - val_loss: 0.3535 - val_accuracy: 0.8494
Epoch 17/100
108/108 [==============================] - 0s 1ms/step - loss: 0.3311 - accuracy: 0.8570 - val_loss: 0.3532 - val_accuracy: 0.8485
Epoch 18/100
108/108 [==============================] - 0s 1ms/step - loss: 0.3247 - accuracy: 0.8612 - val_loss: 0.3525 - val_accuracy: 0.8490
Epoch 19/100
108/108 [==============================] - 0s 1ms/step - loss: 0.3240 - accuracy: 0.8599 - val_loss: 0.3512 - val_accuracy: 0.8494
Epoch 20/100
108/108 [==============================] - 0s 1ms/step - loss: 0.3257 - accuracy: 0.8620 - val_loss: 0.3498 - val_accuracy: 0.8514
Epoch 21/100
108/108 [==============================] - 0s 1ms/step - loss: 0.3244 - accuracy: 0.8586 - val_loss: 0.3511 - val_accuracy: 0.8504
Epoch 22/100
108/108 [==============================] - 0s 1ms/step - loss: 0.3189 - accuracy: 0.8652 - val_loss: 0.3488 - val_accuracy: 0.8514
Epoch 23/100
108/108 [==============================] - 0s 1ms/step - loss: 0.3179 - accuracy: 0.8633 - val_loss: 0.3494 - val_accuracy: 0.8509
Epoch 24/100
108/108 [==============================] - 0s 1ms/step - loss: 0.3153 - accuracy: 0.8609 - val_loss: 0.3483 - val_accuracy: 0.8524
Epoch 25/100
108/108 [==============================] - 0s 1ms/step - loss: 0.3130 - accuracy: 0.8644 - val_loss: 0.3479 - val_accuracy: 0.8533
Epoch 26/100
108/108 [==============================] - 0s 1ms/step - loss: 0.3127 - accuracy: 0.8664 - val_loss: 0.3469 - val_accuracy: 0.8533
Epoch 27/100
108/108 [==============================] - 0s 1ms/step - loss: 0.3101 - accuracy: 0.8729 - val_loss: 0.3474 - val_accuracy: 0.8524
Epoch 28/100
108/108 [==============================] - 0s 1ms/step - loss: 0.3058 - accuracy: 0.8709 - val_loss: 0.3464 - val_accuracy: 0.8504
Epoch 29/100
108/108 [==============================] - 0s 1ms/step - loss: 0.3035 - accuracy: 0.8699 - val_loss: 0.3464 - val_accuracy: 0.8524
Epoch 30/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2995 - accuracy: 0.8726 - val_loss: 0.3460 - val_accuracy: 0.8538
Epoch 31/100
108/108 [==============================] - 0s 1ms/step - loss: 0.3024 - accuracy: 0.8715 - val_loss: 0.3449 - val_accuracy: 0.8562
Epoch 32/100
108/108 [==============================] - 0s 1ms/step - loss: 0.3009 - accuracy: 0.8726 - val_loss: 0.3452 - val_accuracy: 0.8533
Epoch 33/100
108/108 [==============================] - 0s 1ms/step - loss: 0.3005 - accuracy: 0.8735 - val_loss: 0.3450 - val_accuracy: 0.8528
Epoch 34/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2943 - accuracy: 0.8768 - val_loss: 0.3450 - val_accuracy: 0.8558
Epoch 35/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2934 - accuracy: 0.8736 - val_loss: 0.3448 - val_accuracy: 0.8548
Epoch 36/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2897 - accuracy: 0.8805 - val_loss: 0.3446 - val_accuracy: 0.8562
Epoch 37/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2875 - accuracy: 0.8786 - val_loss: 0.3450 - val_accuracy: 0.8558
Epoch 38/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2902 - accuracy: 0.8774 - val_loss: 0.3446 - val_accuracy: 0.8577
Epoch 39/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2848 - accuracy: 0.8789 - val_loss: 0.3450 - val_accuracy: 0.8533
Epoch 40/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2810 - accuracy: 0.8790 - val_loss: 0.3444 - val_accuracy: 0.8558
Epoch 41/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2828 - accuracy: 0.8818 - val_loss: 0.3443 - val_accuracy: 0.8543
Epoch 42/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2800 - accuracy: 0.8848 - val_loss: 0.3434 - val_accuracy: 0.8562
Epoch 43/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2768 - accuracy: 0.8839 - val_loss: 0.3442 - val_accuracy: 0.8558
Epoch 44/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2774 - accuracy: 0.8852 - val_loss: 0.3439 - val_accuracy: 0.8548
Epoch 45/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2714 - accuracy: 0.8842 - val_loss: 0.3438 - val_accuracy: 0.8558
Epoch 46/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2699 - accuracy: 0.8900 - val_loss: 0.3447 - val_accuracy: 0.8543
Epoch 47/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2733 - accuracy: 0.8858 - val_loss: 0.3462 - val_accuracy: 0.8548
Epoch 48/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2700 - accuracy: 0.8880 - val_loss: 0.3439 - val_accuracy: 0.8533
Epoch 49/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2673 - accuracy: 0.8877 - val_loss: 0.3445 - val_accuracy: 0.8577
Epoch 50/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2648 - accuracy: 0.8895 - val_loss: 0.3448 - val_accuracy: 0.8562
Epoch 51/100
108/108 [==============================] - 0s 2ms/step - loss: 0.2631 - accuracy: 0.8913 - val_loss: 0.3454 - val_accuracy: 0.8577
Epoch 52/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2591 - accuracy: 0.8935 - val_loss: 0.3456 - val_accuracy: 0.8567
Epoch 53/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2632 - accuracy: 0.8896 - val_loss: 0.3460 - val_accuracy: 0.8543
Epoch 54/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2535 - accuracy: 0.8944 - val_loss: 0.3468 - val_accuracy: 0.8553
Epoch 55/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2594 - accuracy: 0.8937 - val_loss: 0.3453 - val_accuracy: 0.8558
Epoch 56/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2581 - accuracy: 0.8921 - val_loss: 0.3461 - val_accuracy: 0.8582
Epoch 57/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2475 - accuracy: 0.8998 - val_loss: 0.3479 - val_accuracy: 0.8524
Epoch 58/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2533 - accuracy: 0.8929 - val_loss: 0.3472 - val_accuracy: 0.8572
Epoch 59/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2490 - accuracy: 0.8999 - val_loss: 0.3479 - val_accuracy: 0.8533
Epoch 60/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2485 - accuracy: 0.8955 - val_loss: 0.3472 - val_accuracy: 0.8587
Epoch 61/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2482 - accuracy: 0.8967 - val_loss: 0.3481 - val_accuracy: 0.8567
Epoch 62/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2453 - accuracy: 0.9000 - val_loss: 0.3479 - val_accuracy: 0.8562
Epoch 63/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2412 - accuracy: 0.9032 - val_loss: 0.3513 - val_accuracy: 0.8601
Epoch 64/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2411 - accuracy: 0.9011 - val_loss: 0.3508 - val_accuracy: 0.8558
Epoch 65/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2393 - accuracy: 0.8995 - val_loss: 0.3517 - val_accuracy: 0.8538
Epoch 66/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2335 - accuracy: 0.9060 - val_loss: 0.3509 - val_accuracy: 0.8562
Epoch 67/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2385 - accuracy: 0.9029 - val_loss: 0.3521 - val_accuracy: 0.8562
Epoch 68/100
108/108 [==============================] - 0s 2ms/step - loss: 0.2362 - accuracy: 0.9045 - val_loss: 0.3514 - val_accuracy: 0.8582
Epoch 69/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2349 - accuracy: 0.9019 - val_loss: 0.3527 - val_accuracy: 0.8572
Epoch 70/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2309 - accuracy: 0.9067 - val_loss: 0.3545 - val_accuracy: 0.8558
Epoch 71/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2293 - accuracy: 0.9073 - val_loss: 0.3534 - val_accuracy: 0.8538
Epoch 72/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2290 - accuracy: 0.9095 - val_loss: 0.3549 - val_accuracy: 0.8572
Epoch 73/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2264 - accuracy: 0.9073 - val_loss: 0.3560 - val_accuracy: 0.8548
Epoch 74/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2249 - accuracy: 0.9063 - val_loss: 0.3577 - val_accuracy: 0.8567
Epoch 75/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2280 - accuracy: 0.9058 - val_loss: 0.3569 - val_accuracy: 0.8577
Epoch 76/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2204 - accuracy: 0.9121 - val_loss: 0.3580 - val_accuracy: 0.8558
Epoch 77/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2188 - accuracy: 0.9077 - val_loss: 0.3583 - val_accuracy: 0.8533
Epoch 78/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2155 - accuracy: 0.9119 - val_loss: 0.3587 - val_accuracy: 0.8567
Epoch 79/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2162 - accuracy: 0.9105 - val_loss: 0.3599 - val_accuracy: 0.8538
Epoch 80/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2132 - accuracy: 0.9125 - val_loss: 0.3601 - val_accuracy: 0.8528
Epoch 81/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2097 - accuracy: 0.9166 - val_loss: 0.3649 - val_accuracy: 0.8553
Epoch 82/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2147 - accuracy: 0.9137 - val_loss: 0.3615 - val_accuracy: 0.8538
Epoch 83/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2114 - accuracy: 0.9147 - val_loss: 0.3649 - val_accuracy: 0.8553
Epoch 84/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2143 - accuracy: 0.9135 - val_loss: 0.3611 - val_accuracy: 0.8558
Epoch 85/100
108/108 [==============================] - 0s 2ms/step - loss: 0.2075 - accuracy: 0.9160 - val_loss: 0.3639 - val_accuracy: 0.8553
Epoch 86/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2092 - accuracy: 0.9170 - val_loss: 0.3661 - val_accuracy: 0.8538
Epoch 87/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2073 - accuracy: 0.9157 - val_loss: 0.3679 - val_accuracy: 0.8514
Epoch 88/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2052 - accuracy: 0.9167 - val_loss: 0.3684 - val_accuracy: 0.8572
Epoch 89/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2062 - accuracy: 0.9175 - val_loss: 0.3694 - val_accuracy: 0.8509
Epoch 90/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2062 - accuracy: 0.9175 - val_loss: 0.3703 - val_accuracy: 0.8548
Epoch 91/100
108/108 [==============================] - 0s 1ms/step - loss: 0.1993 - accuracy: 0.9199 - val_loss: 0.3717 - val_accuracy: 0.8494
Epoch 92/100
108/108 [==============================] - 0s 1ms/step - loss: 0.1974 - accuracy: 0.9199 - val_loss: 0.3726 - val_accuracy: 0.8553
Epoch 93/100
108/108 [==============================] - 0s 1ms/step - loss: 0.2013 - accuracy: 0.9183 - val_loss: 0.3691 - val_accuracy: 0.8524
Epoch 94/100
108/108 [==============================] - 0s 1ms/step - loss: 0.1965 - accuracy: 0.9192 - val_loss: 0.3727 - val_accuracy: 0.8509
Epoch 95/100
108/108 [==============================] - 0s 1ms/step - loss: 0.1981 - accuracy: 0.9196 - val_loss: 0.3728 - val_accuracy: 0.8514
Epoch 96/100
108/108 [==============================] - 0s 1ms/step - loss: 0.1976 - accuracy: 0.9211 - val_loss: 0.3740 - val_accuracy: 0.8490
Epoch 97/100
108/108 [==============================] - 0s 1ms/step - loss: 0.1920 - accuracy: 0.9214 - val_loss: 0.3767 - val_accuracy: 0.8577
Epoch 98/100
108/108 [==============================] - 0s 1ms/step - loss: 0.1912 - accuracy: 0.9238 - val_loss: 0.3778 - val_accuracy: 0.8514
Epoch 99/100
108/108 [==============================] - 0s 1ms/step - loss: 0.1930 - accuracy: 0.9224 - val_loss: 0.3773 - val_accuracy: 0.8514
Epoch 100/100
108/108 [==============================] - 0s 1ms/step - loss: 0.1888 - accuracy: 0.9250 - val_loss: 0.3794 - val_accuracy: 0.8509

/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/keras/src/engine/training.py:3000: UserWarning: You are saving your model as an HDF5 file via `model.save()`. This file format is considered legacy. We recommend using instead the native Keras format, e.g. `model.save('my_model.keras')`.
  saving_api.save_model(

checkpoint_cb = keras.callbacks.ModelCheckpoint(filepath="checkpoints/model_epoch={epoch:02d}_val_accuracy={val_accuracy:.2f}.h5")
early_stopping_cb = keras.callbacks.EarlyStopping(patience=5, restore_best_weights=True, verbose=1)

history = model.fit(X_train, Y_train,
                    validation_data=(X_valid, Y_valid), 
                    batch_size = 64,
                    epochs=100, callbacks=[early_stopping_cb, checkpoint_cb])

Epoch 1/100
108/108 [==============================] - 0s 2ms/step - loss: 0.1873 - accuracy: 0.9227 - val_loss: 0.3826 - val_accuracy: 0.8519
Epoch 2/100
108/108 [==============================] - 0s 1ms/step - loss: 0.1916 - accuracy: 0.9217 - val_loss: 0.3801 - val_accuracy: 0.8558
Epoch 3/100
  1/108 [..............................] - ETA: 0s - loss: 0.1274 - accuracy: 0.9688

/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/keras/src/engine/training.py:3000: UserWarning: You are saving your model as an HDF5 file via `model.save()`. This file format is considered legacy. We recommend using instead the native Keras format, e.g. `model.save('my_model.keras')`.
  saving_api.save_model(

108/108 [==============================] - 0s 1ms/step - loss: 0.1870 - accuracy: 0.9254 - val_loss: 0.3804 - val_accuracy: 0.8504
Epoch 4/100
108/108 [==============================] - 0s 1ms/step - loss: 0.1829 - accuracy: 0.9289 - val_loss: 0.3830 - val_accuracy: 0.8524
Epoch 5/100
108/108 [==============================] - 0s 1ms/step - loss: 0.1821 - accuracy: 0.9286 - val_loss: 0.3865 - val_accuracy: 0.8543
Epoch 6/100
108/108 [==============================] - 0s 1ms/step - loss: 0.1872 - accuracy: 0.9247 - val_loss: 0.3864 - val_accuracy: 0.8509
Epoch 7/100
 99/108 [==========================>...] - ETA: 0s - loss: 0.1834 - accuracy: 0.9279Restoring model weights from the end of the best epoch: 2.
108/108 [==============================] - 0s 1ms/step - loss: 0.1825 - accuracy: 0.9278 - val_loss: 0.3858 - val_accuracy: 0.8528
Epoch 7: early stopping

Best Predictive Model

Here we show our final model for the Kaggle Competition

---

ML Unige 2023: Advertisement Impact Prediction

Predicting customer subscriptions from an advertisement campaign

---

https://www.kaggle.com/competitions/ml-unige-2023/submissions

XGBoost with Tuning (No Overfitting)

Introduction to XGBoost in Python. (2020). Quantitative Finance & Algo Trading Blog by QuantInsti. https://blog.quantinsti.com/xgboost-python/

Confusion Matrix

make_confusion_matrix(confusion_matrix(Y_valid, y_pred_final), group_names=['True Neg','False Pos','False Neg','True Pos'],  categories=['Zero', 'One'], cmap=custom_cmap2)

ROC

plot_ROC(Y_valid, y_pred_final)

Kaggle Competition

Best Predictive Model Fitted with 100% Train - XGBoost (Tuning V2)

final_xgb_clf.fit(train_no_subscription,train_only_subscription)

y_test_pred = final_xgb_clf.predict(test_hot) 

Predictions_On_Test = pd.DataFrame({'subscription': y_test_pred}).reset_index()

Predictions_On_Test = Predictions_On_Test.rename(columns={Predictions_On_Test.columns[0]: 'Id'})

pd.DataFrame(Predictions_On_Test).to_csv("Predictions_On_Test_Kaggle.csv",index=False)

print(y_test_pred.shape[0], "predictions correctly exported")

y_test_pred = pd.DataFrame(y_test_pred)
y_test_pred

3837 predictions correctly exported

	0
0	0
1	1
2	0
3	0
4	1
...	...
3832	0
3833	1
3834	0
3835	0
3836	1

3837 rows × 1 columns

---

Public Score (30% Test)

86.446%

---

Best Predictive Model Fitted with 70% Train - XGBoost (Tuning V2)

final_xgb_clf.fit(X_train,Y_train)

y_test_pred = final_xgb_clf.predict(test_hot) 

Predictions_On_Test = pd.DataFrame({'subscription': y_test_pred}).reset_index()

Predictions_On_Test = Predictions_On_Test.rename(columns={Predictions_On_Test.columns[0]: 'Id'})

pd.DataFrame(Predictions_On_Test).to_csv("Predictions_On_Test_Kaggle.csv",index=False)

print(y_test_pred.shape[0], "predictions correctly exported")

y_test_pred = pd.DataFrame(y_test_pred)
y_test_pred

3837 predictions correctly exported

	0
0	0
1	1
2	0
3	0
4	1
...	...
3832	0
3833	1
3834	0
3835	0
3836	1

3837 rows × 1 columns

---

Public Score (30% Test)

86.446%

---